Oct 30 19:46:48.533462 ip-10-0-142-168 systemd[1]: Starting Kubernetes Kubelet... Oct 30 19:46:48.916942 ip-10-0-142-168 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:46:48.916942 ip-10-0-142-168 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 19:46:48.916942 ip-10-0-142-168 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:46:48.916942 ip-10-0-142-168 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:46:48.924594 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924508 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:46:48.924594 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924593 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:46:48.924594 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924598 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924601 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924604 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924607 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924610 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924612 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924615 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924618 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924620 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924623 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924625 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924628 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924630 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924633 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924635 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924638 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924640 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924643 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924645 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924648 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:46:48.924712 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924650 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924653 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924655 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924663 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924668 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924671 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924674 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924677 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924680 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924682 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924685 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924687 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924690 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924693 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924696 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924699 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924701 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924704 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924707 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:46:48.925170 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924709 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924712 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924714 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924717 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924720 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924723 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924726 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924728 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924731 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924734 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924737 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924741 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924743 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924746 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924748 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924751 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924753 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924756 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924759 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:46:48.925630 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924761 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924767 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924769 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924772 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924774 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924777 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924780 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924782 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924786 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924805 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924809 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924812 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924816 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924818 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924821 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924824 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924826 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924829 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924831 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924834 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:46:48.926067 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924836 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924839 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924841 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924845 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924848 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924851 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924853 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924855 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924858 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924861 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924863 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924865 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924868 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924872 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924875 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924878 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924880 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924882 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924885 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924887 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:46:48.926597 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924890 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924893 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924896 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924900 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924904 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.924908 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925690 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925698 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925701 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925704 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925706 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925709 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925712 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925714 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925717 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925719 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925723 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925725 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925728 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:46:48.927059 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925731 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925733 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925736 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925738 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925741 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925744 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925747 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925749 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925752 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925754 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925757 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925759 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925761 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925764 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925767 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925769 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925772 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925774 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925776 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925779 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:46:48.927565 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925781 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925784 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925786 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925789 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925791 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925794 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925797 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925799 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925801 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925804 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925807 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925810 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925813 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925816 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925818 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925821 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925824 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925826 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925829 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:46:48.928037 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925833 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925837 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925839 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925842 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925845 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925848 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925850 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925854 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925857 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925859 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925862 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925864 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925867 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925869 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925872 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925874 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925877 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925879 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925882 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925884 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:46:48.928532 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925886 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925889 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925891 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925894 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925897 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925900 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925902 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925905 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925907 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925909 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925912 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925914 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925917 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925919 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925922 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925924 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925928 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925932 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925937 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:46:48.928991 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925940 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925943 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925945 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925948 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925950 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925954 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925956 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925959 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925961 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925963 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925966 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925968 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925971 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925973 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.925976 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926473 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926480 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926483 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926486 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926489 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:46:48.929464 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926492 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926494 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926497 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926499 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926502 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926504 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926507 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926510 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926512 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926515 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926517 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926521 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926524 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926527 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926529 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926532 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926534 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926537 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926540 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926542 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:46:48.929944 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926545 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926547 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926550 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926552 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926554 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926565 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926569 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926572 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926574 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926578 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926581 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926584 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926587 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926589 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926592 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926594 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926597 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926599 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926602 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:46:48.930444 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926604 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926607 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926610 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926612 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926616 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926618 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926621 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926623 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926626 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926629 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926631 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926634 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926638 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926642 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926645 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926648 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926650 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926653 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926655 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926658 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:46:48.930900 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926660 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926662 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926666 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926669 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926672 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926674 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926677 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926679 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926681 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926684 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926686 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926689 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926691 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926694 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926696 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926699 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926703 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926705 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926710 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:46:48.931481 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926713 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926716 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926718 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926721 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926724 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926726 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926729 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926732 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926734 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926737 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926739 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926742 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926744 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926747 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926749 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926753 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926755 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926758 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926761 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926765 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:46:48.931939 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926767 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926770 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.926772 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926866 2577 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926873 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926879 2577 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926884 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926888 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926891 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926898 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926902 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926906 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926908 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926912 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926915 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926918 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926921 2577 flags.go:64] FLAG: --cgroup-root="" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926924 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926927 2577 flags.go:64] FLAG: --client-ca-file="" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926929 2577 flags.go:64] FLAG: --cloud-provider="external" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926932 2577 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926936 2577 flags.go:64] FLAG: --cluster-domain="" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926939 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 19:46:48.932419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926942 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926945 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926948 2577 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926952 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926956 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926959 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926962 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926965 2577 flags.go:64] FLAG: --contention-profiling="false" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926969 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926972 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926975 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926978 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926982 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926985 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926989 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926992 2577 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926994 2577 flags.go:64] FLAG: --enable-server="true" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.926997 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927005 2577 flags.go:64] FLAG: --event-burst="100" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927009 2577 flags.go:64] FLAG: --event-qps="50" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927012 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927015 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927017 2577 flags.go:64] FLAG: --eviction-hard="" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927021 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 19:46:48.932955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927024 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927027 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927030 2577 flags.go:64] FLAG: --eviction-soft="" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927033 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927036 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927039 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927042 2577 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927044 2577 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927047 2577 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927050 2577 flags.go:64] FLAG: --feature-gates="" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927054 2577 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927057 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927062 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927065 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927068 2577 flags.go:64] FLAG: --healthz-port="10248" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927071 2577 flags.go:64] FLAG: --help="false" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927074 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-142-168.ec2.internal" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927078 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927081 2577 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927084 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927087 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927090 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927093 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 19:46:48.933599 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927096 2577 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927099 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927102 2577 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927106 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927109 2577 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927112 2577 flags.go:64] FLAG: --kube-reserved="" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927115 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927118 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927121 2577 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927123 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927126 2577 flags.go:64] FLAG: --lock-file="" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927129 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927132 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927135 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927140 2577 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927143 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927146 2577 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927149 2577 flags.go:64] FLAG: --logging-format="text" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927152 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927155 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927158 2577 flags.go:64] FLAG: --manifest-url="" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927163 2577 flags.go:64] FLAG: --manifest-url-header="" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927167 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927170 2577 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927174 2577 flags.go:64] FLAG: --max-pods="110" Oct 30 19:46:48.934205 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927177 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927180 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927183 2577 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927186 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927203 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927206 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927209 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927216 2577 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927219 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927222 2577 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927226 2577 flags.go:64] FLAG: --pod-cidr="" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927229 2577 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927232 2577 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927235 2577 flags.go:64] FLAG: --pods-per-core="0" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927238 2577 flags.go:64] FLAG: --port="10250" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927241 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927244 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b2c1ccb4c90b384" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927247 2577 flags.go:64] FLAG: --qos-reserved="" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927249 2577 flags.go:64] FLAG: --read-only-port="10255" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927252 2577 flags.go:64] FLAG: --register-node="true" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927255 2577 flags.go:64] FLAG: --register-with-taints="" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927259 2577 flags.go:64] FLAG: --registry-burst="10" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927262 2577 flags.go:64] FLAG: --registry-qps="5" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927265 2577 flags.go:64] FLAG: --reserved-cpus="" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927267 2577 flags.go:64] FLAG: --reserved-memory="" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927271 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 19:46:48.934825 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927274 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927277 2577 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927281 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927284 2577 flags.go:64] FLAG: --runonce="false" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927286 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927289 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927292 2577 flags.go:64] FLAG: --seccomp-default="false" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927295 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927298 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927301 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927304 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927307 2577 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927310 2577 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927313 2577 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927315 2577 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927318 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927322 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927325 2577 flags.go:64] FLAG: --system-cgroups="" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927328 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927333 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927336 2577 flags.go:64] FLAG: --tls-cert-file="" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927339 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927343 2577 flags.go:64] FLAG: --tls-min-version="" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927346 2577 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927348 2577 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 19:46:48.935439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927351 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927354 2577 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927357 2577 flags.go:64] FLAG: --v="2" Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927361 2577 flags.go:64] FLAG: --version="false" Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927365 2577 flags.go:64] FLAG: --vmodule="" Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927368 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927371 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927529 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927534 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927538 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927541 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927544 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927547 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927549 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927552 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927555 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927558 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927561 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927563 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927566 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927569 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:46:48.936020 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927571 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927574 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927577 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927580 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927583 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927585 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927588 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927590 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927593 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927595 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927598 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927600 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927603 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927605 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927608 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927610 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927612 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927615 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927617 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927620 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:46:48.936568 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927623 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927626 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927630 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927633 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927636 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927639 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927643 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927647 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927650 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927653 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927656 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927658 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927660 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927663 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927666 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927668 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927671 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927674 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927677 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:46:48.937039 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927679 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927681 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927684 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927686 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927689 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927691 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927694 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927696 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927698 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927701 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927703 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927705 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927708 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927711 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927714 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927716 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927719 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927721 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927724 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:46:48.937528 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927726 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927729 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927731 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927733 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927737 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927739 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927742 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927744 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927747 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927749 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927751 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927754 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927756 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927759 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927761 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927764 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927766 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927768 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927771 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:46:48.938664 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927773 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927775 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927778 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927781 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927783 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927785 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927788 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927791 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927794 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927796 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927799 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927801 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927803 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927806 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927808 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.927832 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:46:48.939115 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927866 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927870 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927874 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927876 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927879 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927881 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927884 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927887 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927890 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927892 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927895 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927897 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927899 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927902 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927904 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927906 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927909 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927911 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927914 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:46:48.939561 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927916 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927919 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927921 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927923 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927927 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927930 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927932 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927935 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927937 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927939 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927942 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927944 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927946 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927949 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927951 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927954 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927956 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927960 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927964 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927968 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:46:48.940012 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927972 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927976 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927980 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927983 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927987 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927989 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927992 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927994 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.927997 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928000 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928003 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928005 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928008 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928010 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928013 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928015 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928019 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928021 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928024 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928026 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:46:48.940484 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928029 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928031 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928033 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928036 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928038 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928041 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928044 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928046 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928049 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928052 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928054 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928058 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928061 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928063 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928066 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928068 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928071 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928074 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928077 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:46:48.940947 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928079 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928082 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928084 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928087 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928089 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928091 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928094 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928096 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928099 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928102 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928104 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928107 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928109 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928111 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928114 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928116 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928119 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928121 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928124 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:46:48.941422 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928126 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928128 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928131 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928133 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928136 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928140 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928143 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928145 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.928148 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.935418 2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.935432 2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.935445 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.935449 2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935518 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935523 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935526 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935529 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935532 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935534 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935537 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:46:48.941863 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935540 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935542 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935545 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935547 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935550 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935552 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935555 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935558 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935561 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935563 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935567 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935569 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935572 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935574 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935577 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935579 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935583 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935586 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935588 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935591 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:46:48.942358 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935593 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935596 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935598 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935601 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935603 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935606 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935608 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935611 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935613 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935615 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935618 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935620 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935623 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935625 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935628 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935630 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935632 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935635 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935638 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:46:48.942827 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935640 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935643 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935645 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935648 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935650 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935653 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935656 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935658 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935660 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935663 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935666 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935670 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935674 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935676 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935679 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935682 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935685 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935687 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935690 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:46:48.943419 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935692 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935695 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935697 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935699 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935702 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935704 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935707 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935709 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935712 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935714 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935716 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935719 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935722 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935725 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935727 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935730 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935732 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935735 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935737 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935740 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:46:48.943914 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935742 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935744 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935747 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935749 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935761 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935764 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935766 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935768 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935771 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935773 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935776 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935778 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935781 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935785 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935789 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935792 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935795 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935797 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935800 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:46:48.944394 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935802 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.935805 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.935832 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936024 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936029 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936032 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936035 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936037 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936040 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936042 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936045 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936047 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936050 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936052 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936055 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:46:48.945003 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936057 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936059 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936062 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936071 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936074 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936077 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936080 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936083 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936085 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936088 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936090 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936092 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936095 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936097 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936100 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936102 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936105 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936107 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936109 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936112 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:46:48.945555 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936114 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936116 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936119 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936121 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936124 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936126 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936128 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936131 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936133 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936135 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936138 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936140 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936142 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936145 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936147 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936150 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936157 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936160 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936163 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936166 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:46:48.946078 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936168 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936172 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936175 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936178 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936180 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936183 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936185 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936188 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936207 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936210 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936213 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936215 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936218 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936220 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936223 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936225 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936228 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936230 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936233 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:46:48.946600 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936235 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936238 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936240 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936242 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936245 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936247 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936266 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936270 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936273 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936277 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936287 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936290 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936293 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936296 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936298 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936301 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936304 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936306 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936308 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:46:48.947171 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936311 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936313 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936315 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936318 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936320 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936323 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936325 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936328 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936330 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936333 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936335 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936338 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936340 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936343 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936345 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936348 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:46:48.948689 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.936372 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936415 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936419 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936422 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936425 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936427 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936430 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936432 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936441 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936444 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936450 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936453 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936455 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936458 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936460 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936463 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936465 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936468 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936470 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:46:48.949398 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936472 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936475 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936477 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936480 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936482 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936485 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936487 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936489 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936492 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936494 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936504 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936507 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936509 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936512 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936515 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936517 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936519 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936522 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936524 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936527 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:46:48.950047 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936529 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936532 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936540 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936543 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936546 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936549 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936552 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936556 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936559 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936562 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936564 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936566 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936569 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936571 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936574 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936577 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936579 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936583 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936586 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:46:48.950551 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936588 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936591 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936593 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936596 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936598 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936600 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936603 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936605 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936607 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936610 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936613 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936615 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936618 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936620 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936625 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936629 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936639 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936642 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936645 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:46:48.951074 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936647 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936650 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936653 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936655 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936657 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936660 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936662 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936664 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936667 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936670 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936672 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936675 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936678 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936680 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936683 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936685 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936688 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936690 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936693 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936695 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:46:48.951536 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936697 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936704 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936707 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936710 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936712 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936714 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936717 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936719 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936722 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:48.936726 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.936778 2577 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.939244 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.940331 2577 server.go:1014] "Starting client certificate rotation" Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.940432 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.940470 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.943535 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:46:48.952080 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.949846 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:46:48.962918 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.962902 2577 log.go:25] "Validated CRI v1 runtime API" Oct 30 19:46:48.967541 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.967528 2577 log.go:25] "Validated CRI v1 image API" Oct 30 19:46:48.968391 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.968361 2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 19:46:48.971001 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.970979 2577 fs.go:135] Filesystem UUIDs: map[2c6ad25c-b20b-46c6-9d11-c6c616e59775:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cccf4815-5b4f-4592-a081-750a3f31e5b9:/dev/nvme0n1p4] Oct 30 19:46:48.971096 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.970999 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 19:46:48.976475 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.976366 2577 manager.go:217] Machine: {Timestamp:2026-10-30 19:46:48.975048552 +0000 UTC m=+0.324933362 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100175 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e8f99478f83036e32bd3e5bd4a737 SystemUUID:ec2e8f99-478f-8303-6e32-bd3e5bd4a737 BootID:3c546d73-e57f-4c4b-8c7b-a16667c8f43a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e4:0b:9c:05 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e4:0b:9c:05 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:d0:25:43:53:0e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 19:46:48.976475 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.976471 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 19:46:48.976645 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.976565 2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 19:46:48.979408 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.979387 2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 19:46:48.979539 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.979409 2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 19:46:48.979586 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.979542 2577 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 19:46:48.979586 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.979550 2577 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 19:46:48.979586 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.979574 2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 19:46:48.981452 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.981441 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 19:46:48.982648 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.982637 2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 19:46:48.987344 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.987332 2577 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 19:46:48.987462 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.987453 2577 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 19:46:48.988323 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.988313 2577 kubelet.go:493] "Attempting to sync node with API server" Oct 30 19:46:48.988366 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.988328 2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 19:46:48.988366 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.988339 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 19:46:48.988366 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.988347 2577 kubelet.go:405] "Adding apiserver pod source" Oct 30 19:46:48.988366 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.988356 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 19:46:48.988726 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.988712 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 19:46:48.991284 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:48.991267 2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 19:46:49.000031 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.000009 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 19:46:49.001249 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.001232 2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 19:46:49.001327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.001264 2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 19:46:49.001436 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.001425 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 19:46:49.001477 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.001442 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 19:46:49.001477 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.001447 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 19:46:49.001477 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.001453 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 19:46:49.001477 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.001458 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 19:46:49.001477 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.001465 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 19:46:49.002254 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.002245 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 19:46:49.002254 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.002254 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 19:46:49.002311 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.002262 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 19:46:49.002311 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.002268 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 19:46:49.002311 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.002276 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 19:46:49.002311 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.002284 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 19:46:49.003909 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.003898 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 19:46:49.003974 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.003912 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 19:46:49.007770 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.007755 2577 server.go:1263] "Started kubelet" Oct 30 19:46:49.007857 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.007828 2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 19:46:49.007960 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.007910 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 19:46:49.008011 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.007984 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 19:46:49.009094 ip-10-0-142-168 systemd[1]: Started Kubernetes Kubelet. Oct 30 19:46:49.009419 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.009399 2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 19:46:49.010367 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.010351 2577 server.go:317] "Adding debug handlers to kubelet server" Oct 30 19:46:49.013667 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.013647 2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 19:46:49.016439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.016418 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 19:46:49.016541 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.016428 2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 19:46:49.016541 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.016509 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.017116 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017097 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 19:46:49.017212 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017100 2577 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 19:46:49.017212 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017165 2577 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 19:46:49.017343 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017326 2577 reconstruct.go:97] "Volume reconstruction finished" Oct 30 19:46:49.017451 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017341 2577 reconciler.go:29] "Reconciler: start to sync state" Oct 30 19:46:49.017582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017563 2577 factory.go:55] Registering systemd factory Oct 30 19:46:49.017653 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017588 2577 factory.go:223] Registration of the systemd container factory successfully Oct 30 19:46:49.017815 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017800 2577 factory.go:153] Registering CRI-O factory Oct 30 19:46:49.017857 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017819 2577 factory.go:223] Registration of the crio container factory successfully Oct 30 19:46:49.017895 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017873 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 19:46:49.017895 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017889 2577 factory.go:103] Registering Raw factory Oct 30 19:46:49.017967 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.017902 2577 manager.go:1199] Started watching for new ooms in manager Oct 30 19:46:49.018968 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.018945 2577 manager.go:319] Starting recovery of all containers Oct 30 19:46:49.032110 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.032086 2577 manager.go:324] Recovery completed Oct 30 19:46:49.037273 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.037259 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:46:49.039215 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.039182 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:46:49.039274 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.039228 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:46:49.039274 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.039238 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:46:49.039768 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.039747 2577 cpu_manager.go:226] "Starting" policy="none" Oct 30 19:46:49.039768 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.039757 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:46:49.039897 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.039760 2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 19:46:49.039897 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.039800 2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 19:46:49.040573 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.040554 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:46:49.040648 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.040583 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:46:49.040648 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.040593 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:46:49.041422 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.041400 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 19:46:49.041508 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.041400 2577 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 19:46:49.041966 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.041956 2577 policy_none.go:50] "Start" Oct 30 19:46:49.041995 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.041971 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 19:46:49.041995 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.041981 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 19:46:49.042499 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.041403 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-168.ec2.internal.18e36658fcc83db5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-168.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-168.ec2.internal,},FirstTimestamp:2026-10-30 19:46:49.007726005 +0000 UTC m=+0.357610814,LastTimestamp:2026-10-30 19:46:49.007726005 +0000 UTC m=+0.357610814,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-168.ec2.internal,}" Oct 30 19:46:49.043981 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.043969 2577 policy_none.go:44] "Start" Oct 30 19:46:49.044095 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.044079 2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 19:46:49.044146 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.044135 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dbmtz" Oct 30 19:46:49.049269 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.049249 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dbmtz" Oct 30 19:46:49.055274 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.055252 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 19:46:49.055448 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.055433 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 19:46:49.083835 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.083807 2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.087738 2577 manager.go:341] "Starting Device Plugin manager" Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.087836 2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.087877 2577 server.go:87] "Starting device plugin registration server" Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.088110 2577 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.088122 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.088355 2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.088426 2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.088434 2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 19:46:49.088920 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.088886 2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 19:46:49.108393 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.088930 2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.108393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.102937 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 19:46:49.108393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.104238 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 19:46:49.108393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.104262 2577 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 19:46:49.108393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.104282 2577 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 19:46:49.108393 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.104312 2577 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 19:46:49.134391 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.134376 2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:46:49.137265 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.137247 2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:46:49.188930 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.188867 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:46:49.189699 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.189683 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:46:49.189798 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.189723 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:46:49.189798 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.189738 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:46:49.189798 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.189767 2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.204503 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.204482 2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal"] Oct 30 19:46:49.204578 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.204530 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:46:49.205263 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.205232 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:46:49.205330 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.205266 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:46:49.205330 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.205275 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:46:49.207456 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.207444 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:46:49.207586 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.207573 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.207625 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.207601 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:46:49.208140 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.208125 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:46:49.208212 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.208147 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:46:49.208212 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.208157 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:46:49.208285 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.208126 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:46:49.208285 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.208262 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:46:49.208285 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.208277 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:46:49.209185 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.209170 2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.209246 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.209216 2577 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-168.ec2.internal\": node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.210420 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.210405 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.210477 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.210435 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:46:49.211090 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.211069 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:46:49.211178 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.211103 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:46:49.211178 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.211115 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:46:49.217626 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.217609 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.217701 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.217641 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.217701 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.217689 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41493c3824738811979a28105df74e81-config\") pod \"kube-apiserver-proxy-ip-10-0-142-168.ec2.internal\" (UID: \"41493c3824738811979a28105df74e81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.229699 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.229682 2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-168.ec2.internal\" not found" node="ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.233894 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.233875 2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-168.ec2.internal\" not found" node="ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.254046 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.254024 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.318562 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.318537 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41493c3824738811979a28105df74e81-config\") pod \"kube-apiserver-proxy-ip-10-0-142-168.ec2.internal\" (UID: \"41493c3824738811979a28105df74e81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.318670 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.318566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.318670 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.318587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.318670 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.318618 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.318670 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.318646 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.318670 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.318654 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41493c3824738811979a28105df74e81-config\") pod \"kube-apiserver-proxy-ip-10-0-142-168.ec2.internal\" (UID: \"41493c3824738811979a28105df74e81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.354230 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.354211 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.454701 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.454654 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.534747 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.534728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.538403 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.538378 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:49.555301 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.555283 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.655530 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.655504 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.756022 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.755966 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.856624 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.856598 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:49.940188 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.940053 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 19:46:49.940624 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.940313 2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 19:46:49.940624 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:49.940335 2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 19:46:49.957440 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:49.957413 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:50.053487 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.053425 2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:46:50.054505 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.054462 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:41:50 +0000 UTC" deadline="2028-05-21 03:23:10.430732145 +0000 UTC" Oct 30 19:46:50.054505 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.054499 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13639h36m20.376235656s" Oct 30 19:46:50.058420 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:50.058392 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:50.069640 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:50.069613 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41493c3824738811979a28105df74e81.slice/crio-b6d4cafc1eb47eab5e59ca7ae4f78a7d63dd9850f52f470fa55a6d9aa65185ce WatchSource:0}: Error finding container b6d4cafc1eb47eab5e59ca7ae4f78a7d63dd9850f52f470fa55a6d9aa65185ce: Status 404 returned error can't find the container with id b6d4cafc1eb47eab5e59ca7ae4f78a7d63dd9850f52f470fa55a6d9aa65185ce Oct 30 19:46:50.069876 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:50.069855 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f1ffa9a97654c51d3e03111cf4f982.slice/crio-410c008ae2e899f7593d50532d68df1c269ac4d3982eb1c761d236dbf79cee6d WatchSource:0}: Error finding container 410c008ae2e899f7593d50532d68df1c269ac4d3982eb1c761d236dbf79cee6d: Status 404 returned error can't find the container with id 410c008ae2e899f7593d50532d68df1c269ac4d3982eb1c761d236dbf79cee6d Oct 30 19:46:50.074909 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.074892 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:46:50.107117 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.107072 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" event={"ID":"41493c3824738811979a28105df74e81","Type":"ContainerStarted","Data":"b6d4cafc1eb47eab5e59ca7ae4f78a7d63dd9850f52f470fa55a6d9aa65185ce"} Oct 30 19:46:50.107984 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.107964 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" event={"ID":"22f1ffa9a97654c51d3e03111cf4f982","Type":"ContainerStarted","Data":"410c008ae2e899f7593d50532d68df1c269ac4d3982eb1c761d236dbf79cee6d"} Oct 30 19:46:50.113104 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.113083 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:46:50.113838 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.113820 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:46:50.113913 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.113847 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:46:50.113913 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.113860 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:46:50.158472 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:50.158451 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:50.258924 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:50.258899 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:50.359475 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:50.359401 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:50.460042 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:50.460012 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:50.560767 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:50.560737 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:46:50.613701 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.613621 2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:46:50.616568 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.616539 2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:46:50.641416 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.641391 2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:46:50.641521 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.641501 2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:46:50.665894 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:50.665873 2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:46:52.013764 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.013731 2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:46:52.178498 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.178432 2577 apiserver.go:52] "Watching apiserver" Oct 30 19:46:52.206689 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.206665 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 19:46:52.210411 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.210387 2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 19:46:52.210791 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.210759 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-x62cc","kube-system/global-pull-secret-syncer-jvsfx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd","openshift-cluster-node-tuning-operator/tuned-f2tdf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal","openshift-multus/multus-additional-cni-plugins-gnmqz","openshift-network-diagnostics/network-check-target-bvdgb","kube-system/konnectivity-agent-44glf","kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal","openshift-network-operator/iptables-alerter-9tnc4","openshift-dns/node-resolver-p6pc9","openshift-image-registry/node-ca-ghl6s","openshift-multus/multus-lnxz5","openshift-multus/network-metrics-daemon-lb8sw"] Oct 30 19:46:52.213673 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.213653 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:52.213770 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.213738 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:46:52.215823 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.215802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.218467 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.218084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.220369 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.220348 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.222714 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.222693 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:52.222821 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.222756 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:46:52.224843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.224824 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.228340 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.228314 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-44glf" Oct 30 19:46:52.230762 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.230741 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.233010 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.232992 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.233081 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.233051 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:46:52.235366 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.235284 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 19:46:52.235459 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.235421 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.235527 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.235454 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wgtz9\"" Oct 30 19:46:52.235527 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.235507 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cscqg\"" Oct 30 19:46:52.236822 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.236803 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qlg5x\"" Oct 30 19:46:52.238119 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.238099 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.240350 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.240239 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.241786 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.241765 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n5k26\"" Oct 30 19:46:52.244615 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.244597 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 19:46:52.248442 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.248425 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-bvhff\"" Oct 30 19:46:52.248522 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.248472 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-h4pgj\"" Oct 30 19:46:52.260147 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.260126 2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 19:46:52.260253 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.260189 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ks25h" Oct 30 19:46:52.269178 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.269161 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ks25h" Oct 30 19:46:52.307802 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.307709 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 19:46:52.318098 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-conf-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.318194 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318132 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s49fn\" (UniqueName: \"kubernetes.io/projected/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-kube-api-access-s49fn\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.318194 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318164 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-dbus\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.318194 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm7d6\" (UniqueName: \"kubernetes.io/projected/018f82d1-73aa-4c50-bce5-6fef86b947b0-kube-api-access-sm7d6\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.318327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318203 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-bin\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.318327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318224 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3009ff95-7457-4856-be99-e3891d923002-agent-certs\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:46:52.318327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318244 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cnibin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.318327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318257 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-multus\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.318327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318271 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-device-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.318327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318288 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.318327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-host\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.318327 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-var-lib-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-systemd\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-run\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318393 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-tuned\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318410 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-system-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318431 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-bin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318450 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3864b62c-1c37-4c73-89c5-136a1f402078-iptables-alerter-script\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318496 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpw8x\" (UniqueName: \"kubernetes.io/projected/bb90a295-9822-4874-9014-0a91ac7a73c5-kube-api-access-kpw8x\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318530 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-tmp\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318555 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3009ff95-7457-4856-be99-e3891d923002-konnectivity-ca\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318585 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-slash\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318601 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-sys\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-daemon-config\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.318641 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318643 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-etc-kubernetes\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318663 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-serviceca\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318677 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318693 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-var-lib-kubelet\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkzlx\" (UniqueName: \"kubernetes.io/projected/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-kube-api-access-vkzlx\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-kubelet\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-multus-certs\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318791 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-socket-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318805 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-node-log\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-netd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-system-cni-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318862 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318915 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318933 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4w9j\" (UniqueName: \"kubernetes.io/projected/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-kube-api-access-f4w9j\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-kubelet-config\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.319221 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318975 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3864b62c-1c37-4c73-89c5-136a1f402078-host-slash\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.318998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-netns\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-kubernetes\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319028 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm4wj\" (UniqueName: \"kubernetes.io/projected/3864b62c-1c37-4c73-89c5-136a1f402078-kube-api-access-hm4wj\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319111 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-cnibin\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319132 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-config\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysconfig\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319187 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-conf\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-host\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319227 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-systemd-units\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-env-overrides\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319273 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-modprobe-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-k8s-cni-cncf-io\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:52.319829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319331 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-systemd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319377 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319401 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cni-binary-copy\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-socket-dir-parent\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319437 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb90a295-9822-4874-9014-0a91ac7a73c5-hosts-file\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-kubelet\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319487 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzn4q\" (UniqueName: \"kubernetes.io/projected/e1e0c2b5-3116-4963-b1df-fef06ea58937-kube-api-access-fzn4q\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319523 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-lib-modules\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-netns\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-hostroot\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319595 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb90a295-9822-4874-9014-0a91ac7a73c5-tmp-dir\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-etc-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319635 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovn-node-metrics-cert\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319660 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-script-lib\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319685 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:52.320481 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-registration-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.321207 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-sys-fs\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.321207 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319771 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbgzs\" (UniqueName: \"kubernetes.io/projected/62561598-09a8-4cb2-b04d-81ce7c0b8411-kube-api-access-nbgzs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:52.321207 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grrnx\" (UniqueName: \"kubernetes.io/projected/298e0d65-779b-4fd0-8931-43596a15d5fa-kube-api-access-grrnx\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.321207 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-ovn\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.321207 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-log-socket\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.321207 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.319876 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.329663 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329621 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 19:46:52.329663 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329635 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 19:46:52.329663 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329648 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 19:46:52.329843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329685 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 19:46:52.329843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329637 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 19:46:52.329843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329773 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 19:46:52.329843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329774 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 19:46:52.329843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329822 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 19:46:52.329843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329827 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 19:46:52.329843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329832 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 19:46:52.329843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329845 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329856 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329868 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329883 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329930 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329931 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329951 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329960 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329962 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.329986 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.330144 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.330144 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.330216 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:46:52.330232 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.330232 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 19:46:52.336391 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.336354 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 19:46:52.336798 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.336778 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-grfkh\"" Oct 30 19:46:52.336798 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.336786 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d5xr8\"" Oct 30 19:46:52.336942 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.336798 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-97cb9\"" Oct 30 19:46:52.420744 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.420712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-systemd\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.420744 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.420749 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-run\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.420981 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.420939 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-systemd\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421036 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.420979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-tuned\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421036 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421006 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-run\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421036 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421010 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-system-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421036 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421035 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-bin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421053 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3864b62c-1c37-4c73-89c5-136a1f402078-iptables-alerter-script\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421068 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpw8x\" (UniqueName: \"kubernetes.io/projected/bb90a295-9822-4874-9014-0a91ac7a73c5-kube-api-access-kpw8x\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-tmp\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421124 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-bin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421133 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3009ff95-7457-4856-be99-e3891d923002-konnectivity-ca\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-slash\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421176 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-sys\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421196 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421181 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-system-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-daemon-config\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421203 2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-etc-kubernetes\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421266 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-serviceca\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421291 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-var-lib-kubelet\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421337 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkzlx\" (UniqueName: \"kubernetes.io/projected/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-kube-api-access-vkzlx\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421358 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-kubelet\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421383 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-multus-certs\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-socket-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.421443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-node-log\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-netd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-system-cni-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421503 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421528 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421569 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-etc-kubernetes\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421579 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4w9j\" (UniqueName: \"kubernetes.io/projected/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-kube-api-access-f4w9j\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421605 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-kubelet-config\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421629 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3864b62c-1c37-4c73-89c5-136a1f402078-host-slash\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-netns\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421687 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-kubernetes\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421692 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421710 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm4wj\" (UniqueName: \"kubernetes.io/projected/3864b62c-1c37-4c73-89c5-136a1f402078-kube-api-access-hm4wj\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421785 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-cnibin\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.421847 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421813 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-config\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421839 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysconfig\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421857 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-netns\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421864 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-conf\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421889 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-host\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421936 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-systemd-units\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421958 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-env-overrides\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421980 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-modprobe-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-k8s-cni-cncf-io\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422036 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422056 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-systemd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422104 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421237 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cni-binary-copy\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-socket-dir-parent\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.422684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422180 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb90a295-9822-4874-9014-0a91ac7a73c5-hosts-file\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-kubelet\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422228 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzn4q\" (UniqueName: \"kubernetes.io/projected/e1e0c2b5-3116-4963-b1df-fef06ea58937-kube-api-access-fzn4q\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-lib-modules\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422267 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-netns\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-hostroot\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422339 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-hostroot\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422355 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-node-log\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422374 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-kubernetes\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422421 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-netd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422444 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-kubelet-config\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.422448 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422471 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-system-cni-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb90a295-9822-4874-9014-0a91ac7a73c5-tmp-dir\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422523 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-serviceca\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422539 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-etc-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422563 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovn-node-metrics-cert\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422594 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-slash\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.423523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422631 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3864b62c-1c37-4c73-89c5-136a1f402078-host-slash\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.422635 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret podName:a3757b61-1b91-4739-a7fc-ecbfcb4feea6 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:52.922575566 +0000 UTC m=+3.082756747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret") pod "global-pull-secret-syncer-jvsfx" (UID: "a3757b61-1b91-4739-a7fc-ecbfcb4feea6") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-script-lib\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422682 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-sys\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422692 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421569 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-var-lib-kubelet\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422727 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-kubelet\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422734 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-registration-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422763 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-sys-fs\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422808 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-multus-certs\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422807 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbgzs\" (UniqueName: \"kubernetes.io/projected/62561598-09a8-4cb2-b04d-81ce7c0b8411-kube-api-access-nbgzs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422841 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-etc-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422853 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grrnx\" (UniqueName: \"kubernetes.io/projected/298e0d65-779b-4fd0-8931-43596a15d5fa-kube-api-access-grrnx\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422888 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.422956 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-cnibin\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.424352 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423010 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423114 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-registration-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423179 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423243 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423333 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-sys-fs\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421813 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3009ff95-7457-4856-be99-e3891d923002-konnectivity-ca\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423366 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb90a295-9822-4874-9014-0a91ac7a73c5-tmp-dir\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423410 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-netns\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423435 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-lib-modules\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423446 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysconfig\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423475 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-systemd-units\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423480 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-host\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423512 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-socket-dir-parent\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423530 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-daemon-config\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423538 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-conf\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.421814 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3864b62c-1c37-4c73-89c5-136a1f402078-iptables-alerter-script\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423541 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb90a295-9822-4874-9014-0a91ac7a73c5-hosts-file\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423592 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-kubelet\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425067 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.423603 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423601 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-systemd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423668 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-modprobe-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-ovn\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423959 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-ovn\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-log-socket\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.424024 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs podName:62561598-09a8-4cb2-b04d-81ce7c0b8411 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:52.924011818 +0000 UTC m=+3.084192992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs") pod "network-metrics-daemon-lb8sw" (UID: "62561598-09a8-4cb2-b04d-81ce7c0b8411") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.423543 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-k8s-cni-cncf-io\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424041 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cni-binary-copy\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424010 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424063 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424088 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-conf-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s49fn\" (UniqueName: \"kubernetes.io/projected/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-kube-api-access-s49fn\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-dbus\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm7d6\" (UniqueName: \"kubernetes.io/projected/018f82d1-73aa-4c50-bce5-6fef86b947b0-kube-api-access-sm7d6\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424218 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-bin\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424243 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3009ff95-7457-4856-be99-e3891d923002-agent-certs\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:46:52.425582 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424277 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cnibin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-multus\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424329 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-device-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424381 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-host\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424405 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-var-lib-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424498 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-device-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424507 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-var-lib-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424562 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cnibin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424603 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-multus\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424838 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-dbus\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424166 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-socket-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424848 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-config\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424161 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-log-socket\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424912 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-bin\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424945 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-host\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.426111 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424992 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.426785 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424991 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.426785 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.424838 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-script-lib\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.426785 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.425221 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-tmp\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.426785 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.425290 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-conf-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.426785 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.425996 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-tuned\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.426785 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.426183 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-env-overrides\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.427271 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.427245 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3009ff95-7457-4856-be99-e3891d923002-agent-certs\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:46:52.431060 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.430985 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovn-node-metrics-cert\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.451178 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.451149 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkzlx\" (UniqueName: \"kubernetes.io/projected/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-kube-api-access-vkzlx\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.451546 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.451525 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4w9j\" (UniqueName: \"kubernetes.io/projected/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-kube-api-access-f4w9j\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.454540 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.454031 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm4wj\" (UniqueName: \"kubernetes.io/projected/3864b62c-1c37-4c73-89c5-136a1f402078-kube-api-access-hm4wj\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.454540 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.454087 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:46:52.454540 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.454108 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:46:52.454540 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.454121 2577 projected.go:196] Error preparing data for projected volume kube-api-access-wdmfq for pod openshift-network-diagnostics/network-check-target-bvdgb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:46:52.454540 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.454178 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq podName:da64e35d-dc35-4bc8-89cb-23a7eaf75836 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:52.95416032 +0000 UTC m=+3.114341510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wdmfq" (UniqueName: "kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq") pod "network-check-target-bvdgb" (UID: "da64e35d-dc35-4bc8-89cb-23a7eaf75836") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:46:52.454773 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.454588 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzn4q\" (UniqueName: \"kubernetes.io/projected/e1e0c2b5-3116-4963-b1df-fef06ea58937-kube-api-access-fzn4q\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.454773 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.454759 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s49fn\" (UniqueName: \"kubernetes.io/projected/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-kube-api-access-s49fn\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.455568 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.455544 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpw8x\" (UniqueName: \"kubernetes.io/projected/bb90a295-9822-4874-9014-0a91ac7a73c5-kube-api-access-kpw8x\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.457017 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.456998 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbgzs\" (UniqueName: \"kubernetes.io/projected/62561598-09a8-4cb2-b04d-81ce7c0b8411-kube-api-access-nbgzs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:52.457161 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.457141 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm7d6\" (UniqueName: \"kubernetes.io/projected/018f82d1-73aa-4c50-bce5-6fef86b947b0-kube-api-access-sm7d6\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.457373 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.457358 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grrnx\" (UniqueName: \"kubernetes.io/projected/298e0d65-779b-4fd0-8931-43596a15d5fa-kube-api-access-grrnx\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.530307 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.530284 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:46:52.541377 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.541352 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:46:52.541642 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:52.541617 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb90a295_9822_4874_9014_0a91ac7a73c5.slice/crio-67aca29c1952f4205820a503a15429729f5c93cee8bfbb5ca5e638d824be5ebc WatchSource:0}: Error finding container 67aca29c1952f4205820a503a15429729f5c93cee8bfbb5ca5e638d824be5ebc: Status 404 returned error can't find the container with id 67aca29c1952f4205820a503a15429729f5c93cee8bfbb5ca5e638d824be5ebc Oct 30 19:46:52.549492 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.549468 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-44glf" Oct 30 19:46:52.554215 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.554195 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:46:52.560671 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:52.560645 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3009ff95_7457_4856_be99_e3891d923002.slice/crio-1742198c1994027a8d5414071eef86a8e5ca5784dcdc1cad0a3052620a232b23 WatchSource:0}: Error finding container 1742198c1994027a8d5414071eef86a8e5ca5784dcdc1cad0a3052620a232b23: Status 404 returned error can't find the container with id 1742198c1994027a8d5414071eef86a8e5ca5784dcdc1cad0a3052620a232b23 Oct 30 19:46:52.564284 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:52.564250 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc3812fa_69b5_4e6e_bca9_91a3b191ed32.slice/crio-6203dbb7777cbe89c53a06f754fd24c2525d8884391af7b32e157065d225c9b5 WatchSource:0}: Error finding container 6203dbb7777cbe89c53a06f754fd24c2525d8884391af7b32e157065d225c9b5: Status 404 returned error can't find the container with id 6203dbb7777cbe89c53a06f754fd24c2525d8884391af7b32e157065d225c9b5 Oct 30 19:46:52.575323 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.575158 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lnxz5" Oct 30 19:46:52.577472 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.577453 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:46:52.579039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.579021 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:46:52.591507 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:52.591475 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8d4383_c954_4fed_aadf_f1ee7c0eb24e.slice/crio-4b96e6b037426a85d3a891398d5ccfa339192e04ce18ad4402cb8f4a9b6c81bd WatchSource:0}: Error finding container 4b96e6b037426a85d3a891398d5ccfa339192e04ce18ad4402cb8f4a9b6c81bd: Status 404 returned error can't find the container with id 4b96e6b037426a85d3a891398d5ccfa339192e04ce18ad4402cb8f4a9b6c81bd Oct 30 19:46:52.592387 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:52.592365 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298e0d65_779b_4fd0_8931_43596a15d5fa.slice/crio-979ecd3c5b31a7cde5cc776ac33c2cfa5c47782725662283a7f08baa058fc829 WatchSource:0}: Error finding container 979ecd3c5b31a7cde5cc776ac33c2cfa5c47782725662283a7f08baa058fc829: Status 404 returned error can't find the container with id 979ecd3c5b31a7cde5cc776ac33c2cfa5c47782725662283a7f08baa058fc829 Oct 30 19:46:52.597208 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.597162 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:46:52.604430 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.604282 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:46:52.607922 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:52.607870 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8ee0f0_ae69_46d4_985f_b8c9e5f5ac57.slice/crio-2d70e83de570fe12631bdfc08c7f42ccef4b37586478f4045968742581a1bc44 WatchSource:0}: Error finding container 2d70e83de570fe12631bdfc08c7f42ccef4b37586478f4045968742581a1bc44: Status 404 returned error can't find the container with id 2d70e83de570fe12631bdfc08c7f42ccef4b37586478f4045968742581a1bc44 Oct 30 19:46:52.613628 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:46:52.613605 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e0c2b5_3116_4963_b1df_fef06ea58937.slice/crio-d70ff94cd072c48e42ad24591d42f075373f1326410d83b4e8b19d99a659c1c1 WatchSource:0}: Error finding container d70ff94cd072c48e42ad24591d42f075373f1326410d83b4e8b19d99a659c1c1: Status 404 returned error can't find the container with id d70ff94cd072c48e42ad24591d42f075373f1326410d83b4e8b19d99a659c1c1 Oct 30 19:46:52.725062 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.725042 2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:46:52.927833 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.927799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:52.928021 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:52.927877 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:52.928100 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.928072 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:52.928150 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.928127 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs podName:62561598-09a8-4cb2-b04d-81ce7c0b8411 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:53.928110595 +0000 UTC m=+4.088291785 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs") pod "network-metrics-daemon-lb8sw" (UID: "62561598-09a8-4cb2-b04d-81ce7c0b8411") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:52.928532 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.928448 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:52.928532 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:52.928501 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret podName:a3757b61-1b91-4739-a7fc-ecbfcb4feea6 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:53.928485663 +0000 UTC m=+4.088666847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret") pod "global-pull-secret-syncer-jvsfx" (UID: "a3757b61-1b91-4739-a7fc-ecbfcb4feea6") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:53.031268 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.031161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:53.031561 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:53.031328 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:46:53.031561 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:53.031348 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:46:53.031561 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:53.031360 2577 projected.go:196] Error preparing data for projected volume kube-api-access-wdmfq for pod openshift-network-diagnostics/network-check-target-bvdgb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:46:53.031561 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:53.031412 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq podName:da64e35d-dc35-4bc8-89cb-23a7eaf75836 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:54.031396863 +0000 UTC m=+4.191578043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdmfq" (UniqueName: "kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq") pod "network-check-target-bvdgb" (UID: "da64e35d-dc35-4bc8-89cb-23a7eaf75836") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:46:53.271784 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.271717 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:41:52 +0000 UTC" deadline="2028-06-20 02:27:41.776288444 +0000 UTC" Oct 30 19:46:53.271784 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.271780 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14358h40m48.504512324s" Oct 30 19:46:53.306525 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.306359 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" event={"ID":"dc3812fa-69b5-4e6e-bca9-91a3b191ed32","Type":"ContainerStarted","Data":"6203dbb7777cbe89c53a06f754fd24c2525d8884391af7b32e157065d225c9b5"} Oct 30 19:46:53.309263 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.309155 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-44glf" event={"ID":"3009ff95-7457-4856-be99-e3891d923002","Type":"ContainerStarted","Data":"1742198c1994027a8d5414071eef86a8e5ca5784dcdc1cad0a3052620a232b23"} Oct 30 19:46:53.317069 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.317043 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" event={"ID":"018f82d1-73aa-4c50-bce5-6fef86b947b0","Type":"ContainerStarted","Data":"108e4ff9a325e8336bd5555eff508483b2a9634189a3084567c4e224fd0bb00d"} Oct 30 19:46:53.321151 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.321125 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p6pc9" event={"ID":"bb90a295-9822-4874-9014-0a91ac7a73c5","Type":"ContainerStarted","Data":"67aca29c1952f4205820a503a15429729f5c93cee8bfbb5ca5e638d824be5ebc"} Oct 30 19:46:53.325922 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.325879 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" event={"ID":"41493c3824738811979a28105df74e81","Type":"ContainerStarted","Data":"a5706c86f4ee1ac55bfeec1a16cdd4d6ae2fbfaa77505814e417aa98c997eb61"} Oct 30 19:46:53.331222 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.330671 2577 generic.go:356] "Generic (PLEG): container finished" podID="22f1ffa9a97654c51d3e03111cf4f982" containerID="93fd4701257dcad2a38712b9fa5b55ac6b15e96fba5f350ed25089b16b851392" exitCode=0 Oct 30 19:46:53.331222 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.330750 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" event={"ID":"22f1ffa9a97654c51d3e03111cf4f982","Type":"ContainerDied","Data":"93fd4701257dcad2a38712b9fa5b55ac6b15e96fba5f350ed25089b16b851392"} Oct 30 19:46:53.343078 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.343054 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"d70ff94cd072c48e42ad24591d42f075373f1326410d83b4e8b19d99a659c1c1"} Oct 30 19:46:53.355683 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.355657 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ghl6s" event={"ID":"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57","Type":"ContainerStarted","Data":"2d70e83de570fe12631bdfc08c7f42ccef4b37586478f4045968742581a1bc44"} Oct 30 19:46:53.363584 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.363559 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9tnc4" event={"ID":"3864b62c-1c37-4c73-89c5-136a1f402078","Type":"ContainerStarted","Data":"66c35a3180c32bebd160df0dfb67293ffaac1bb6aea0e22cd499740e35bad0d4"} Oct 30 19:46:53.366010 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.365984 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerStarted","Data":"979ecd3c5b31a7cde5cc776ac33c2cfa5c47782725662283a7f08baa058fc829"} Oct 30 19:46:53.370665 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.370603 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lnxz5" event={"ID":"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e","Type":"ContainerStarted","Data":"4b96e6b037426a85d3a891398d5ccfa339192e04ce18ad4402cb8f4a9b6c81bd"} Oct 30 19:46:53.414057 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.413999 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" podStartSLOduration=2.413981247 podStartE2EDuration="2.413981247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:53.36929002 +0000 UTC m=+3.529471218" watchObservedRunningTime="2026-10-30 19:46:53.413981247 +0000 UTC m=+3.574162445" Oct 30 19:46:53.941203 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.940863 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:53.941203 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:53.941123 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:53.941203 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:53.941155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:53.941509 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:53.941207 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret podName:a3757b61-1b91-4739-a7fc-ecbfcb4feea6 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:55.941179674 +0000 UTC m=+6.101360873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret") pod "global-pull-secret-syncer-jvsfx" (UID: "a3757b61-1b91-4739-a7fc-ecbfcb4feea6") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:53.941509 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:53.941268 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:53.941509 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:53.941318 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs podName:62561598-09a8-4cb2-b04d-81ce7c0b8411 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:55.941301818 +0000 UTC m=+6.101483000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs") pod "network-metrics-daemon-lb8sw" (UID: "62561598-09a8-4cb2-b04d-81ce7c0b8411") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:54.041620 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:54.041583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:54.041800 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:54.041784 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:46:54.041856 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:54.041812 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:46:54.041856 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:54.041826 2577 projected.go:196] Error preparing data for projected volume kube-api-access-wdmfq for pod openshift-network-diagnostics/network-check-target-bvdgb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:46:54.041977 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:54.041883 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq podName:da64e35d-dc35-4bc8-89cb-23a7eaf75836 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:56.04186569 +0000 UTC m=+6.202046871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdmfq" (UniqueName: "kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq") pod "network-check-target-bvdgb" (UID: "da64e35d-dc35-4bc8-89cb-23a7eaf75836") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:46:54.272514 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:54.272398 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:41:52 +0000 UTC" deadline="2028-04-11 11:48:52.239497114 +0000 UTC" Oct 30 19:46:54.272514 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:54.272437 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12688h1m57.967063906s" Oct 30 19:46:54.299607 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:54.296749 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:54.299607 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:54.296876 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:46:54.299607 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:54.297290 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:54.299607 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:54.297399 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:46:54.299607 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:54.297467 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:54.299607 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:54.297528 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:46:54.401719 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:54.401686 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" event={"ID":"22f1ffa9a97654c51d3e03111cf4f982","Type":"ContainerStarted","Data":"8883777d502bdb5c79032cc3a3abe5bef45bada191ce3491fe3bd5e203c5d83b"} Oct 30 19:46:55.417450 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:55.415995 2577 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="048e2738ddedbdfe5c782a354b92f182b5880d0a0e639a6c998eaa9b5911d5ff" exitCode=0 Oct 30 19:46:55.417450 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:55.417207 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerDied","Data":"048e2738ddedbdfe5c782a354b92f182b5880d0a0e639a6c998eaa9b5911d5ff"} Oct 30 19:46:55.495188 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:55.495126 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" podStartSLOduration=4.495108452 podStartE2EDuration="4.495108452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:54.446425734 +0000 UTC m=+4.606607121" watchObservedRunningTime="2026-10-30 19:46:55.495108452 +0000 UTC m=+5.655289650" Oct 30 19:46:55.957992 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:55.957099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:55.957992 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:55.957153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:55.957992 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:55.957374 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:55.957992 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:55.957463 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs podName:62561598-09a8-4cb2-b04d-81ce7c0b8411 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:59.957443982 +0000 UTC m=+10.117625176 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs") pod "network-metrics-daemon-lb8sw" (UID: "62561598-09a8-4cb2-b04d-81ce7c0b8411") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:55.957992 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:55.957827 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:55.957992 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:55.957873 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret podName:a3757b61-1b91-4739-a7fc-ecbfcb4feea6 nodeName:}" failed. No retries permitted until 2026-10-30 19:46:59.957858208 +0000 UTC m=+10.118039397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret") pod "global-pull-secret-syncer-jvsfx" (UID: "a3757b61-1b91-4739-a7fc-ecbfcb4feea6") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:56.057911 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:56.057863 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:56.058119 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:56.058101 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:46:56.058210 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:56.058128 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:46:56.058210 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:56.058141 2577 projected.go:196] Error preparing data for projected volume kube-api-access-wdmfq for pod openshift-network-diagnostics/network-check-target-bvdgb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:46:56.058210 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:56.058200 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq podName:da64e35d-dc35-4bc8-89cb-23a7eaf75836 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:00.058181996 +0000 UTC m=+10.218363171 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdmfq" (UniqueName: "kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq") pod "network-check-target-bvdgb" (UID: "da64e35d-dc35-4bc8-89cb-23a7eaf75836") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:46:56.306872 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:56.306807 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:56.307480 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:56.307450 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:46:56.307962 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:56.307942 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:56.308089 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:56.308068 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:46:56.308177 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:56.308152 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:56.308301 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:56.308267 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:46:57.428568 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:57.428528 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-44glf" event={"ID":"3009ff95-7457-4856-be99-e3891d923002","Type":"ContainerStarted","Data":"7a18cb206a2c3d11a219b622920e80b0c8f84d488ac29514ff97bc73e8dba0e7"} Oct 30 19:46:58.297737 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:58.296820 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:46:58.297737 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:58.296982 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:46:58.297737 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:58.297390 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:58.297737 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:58.297480 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:46:58.297737 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:58.297532 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:58.297737 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:58.297610 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:46:58.440023 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:58.439983 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ghl6s" event={"ID":"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57","Type":"ContainerStarted","Data":"ecf57481361bdcf9f892059f0146af8632615ff0dcf41fc378138c58792a80e4"} Oct 30 19:46:58.487088 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:58.485994 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-44glf" podStartSLOduration=4.659382018 podStartE2EDuration="8.485975674s" totalImagesPullingTime="3.826593656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:57.473868554 +0000 UTC m=+7.634049751" watchObservedRunningTime="2026-10-30 19:46:58.485975674 +0000 UTC m=+8.646156873" Oct 30 19:46:58.487088 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:58.486236 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ghl6s" podStartSLOduration=3.449676609 podStartE2EDuration="8.486229772s" totalImagesPullingTime="5.036553163s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:58.485077216 +0000 UTC m=+8.645258417" watchObservedRunningTime="2026-10-30 19:46:58.486229772 +0000 UTC m=+8.646410971" Oct 30 19:46:59.446085 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:59.445955 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" event={"ID":"018f82d1-73aa-4c50-bce5-6fef86b947b0","Type":"ContainerStarted","Data":"1167109d174f70a10ff4b1ff7c4907f3d009eb6a9df0bafe4d0a678de535290d"} Oct 30 19:46:59.992811 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:59.992777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:46:59.992962 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:46:59.992836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:46:59.992962 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:59.992917 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:59.992962 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:59.992954 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:46:59.993114 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:59.992977 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret podName:a3757b61-1b91-4739-a7fc-ecbfcb4feea6 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:07.992959574 +0000 UTC m=+18.153140767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret") pod "global-pull-secret-syncer-jvsfx" (UID: "a3757b61-1b91-4739-a7fc-ecbfcb4feea6") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:46:59.993114 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:46:59.992995 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs podName:62561598-09a8-4cb2-b04d-81ce7c0b8411 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:07.992986419 +0000 UTC m=+18.153167595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs") pod "network-metrics-daemon-lb8sw" (UID: "62561598-09a8-4cb2-b04d-81ce7c0b8411") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:47:00.094280 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.094243 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:00.094433 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:00.094414 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:47:00.094475 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:00.094446 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:47:00.094475 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:00.094460 2577 projected.go:196] Error preparing data for projected volume kube-api-access-wdmfq for pod openshift-network-diagnostics/network-check-target-bvdgb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:47:00.094531 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:00.094520 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq podName:da64e35d-dc35-4bc8-89cb-23a7eaf75836 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:08.094499645 +0000 UTC m=+18.254680823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdmfq" (UniqueName: "kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq") pod "network-check-target-bvdgb" (UID: "da64e35d-dc35-4bc8-89cb-23a7eaf75836") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:47:00.157344 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.157248 2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 19:47:00.287876 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.287778 2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:47:00.157270605Z","UUID":"e66b1a91-8321-46e6-ba4a-af4d446587f7","Handler":null,"Name":"","Endpoint":""} Oct 30 19:47:00.290416 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.290392 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 19:47:00.290528 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.290426 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 19:47:00.295177 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.295154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:00.295287 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.295188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:00.295287 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:00.295264 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:00.295581 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.295447 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:00.295581 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:00.295547 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:00.296126 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:00.296100 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:00.451937 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.450087 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerStarted","Data":"81704250eeca65fe674598dc5d1b23d2233cf2ab680ec669895eb806f8b25a5a"} Oct 30 19:47:00.452555 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.452528 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lnxz5" event={"ID":"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e","Type":"ContainerStarted","Data":"13f7ad8cf083b251af7c177588b0f4c4a4d985a961d0da322c59691273b7763c"} Oct 30 19:47:00.454737 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.454698 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" event={"ID":"018f82d1-73aa-4c50-bce5-6fef86b947b0","Type":"ContainerStarted","Data":"f8407a9d2f1708763d96a1425b1f60bf5cb7dc33d18feb2b55a6661881d7d493"} Oct 30 19:47:00.566534 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:00.566301 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-lnxz5" podStartSLOduration=3.251971476 podStartE2EDuration="10.566284426s" totalImagesPullingTime="7.31431295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:00.565389519 +0000 UTC m=+10.725570716" watchObservedRunningTime="2026-10-30 19:47:00.566284426 +0000 UTC m=+10.726465623" Oct 30 19:47:01.066153 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:01.066115 2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-44glf" Oct 30 19:47:01.067340 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:01.067311 2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-44glf" Oct 30 19:47:01.459353 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:01.459273 2577 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="81704250eeca65fe674598dc5d1b23d2233cf2ab680ec669895eb806f8b25a5a" exitCode=0 Oct 30 19:47:01.459779 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:01.459378 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerDied","Data":"81704250eeca65fe674598dc5d1b23d2233cf2ab680ec669895eb806f8b25a5a"} Oct 30 19:47:01.461970 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:01.461939 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" event={"ID":"018f82d1-73aa-4c50-bce5-6fef86b947b0","Type":"ContainerStarted","Data":"86b8be424296d414001253aed498f4248ff139dbc878a570673322c04768e8d9"} Oct 30 19:47:01.463935 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:01.463649 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p6pc9" event={"ID":"bb90a295-9822-4874-9014-0a91ac7a73c5","Type":"ContainerStarted","Data":"14ad31d7f877f0387f042f13cb979be5b717d4ac1bc12e9476d3d03036f7c39f"} Oct 30 19:47:01.554135 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:01.554077 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-p6pc9" podStartSLOduration=3.615428719 podStartE2EDuration="11.554058958s" totalImagesPullingTime="7.938630239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:01.55350243 +0000 UTC m=+11.713683629" watchObservedRunningTime="2026-10-30 19:47:01.554058958 +0000 UTC m=+11.714240155" Oct 30 19:47:02.295095 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.294819 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:02.295248 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.294821 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:02.295248 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:02.295158 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:02.295248 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:02.295192 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:02.295248 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.294857 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:02.295448 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:02.295273 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:02.467433 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.467396 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9tnc4" event={"ID":"3864b62c-1c37-4c73-89c5-136a1f402078","Type":"ContainerStarted","Data":"ab7864ce89535d5da7778c0ad5e57a318ed6b52b1a1e54bf89863a45b22d3b4b"} Oct 30 19:47:02.469647 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.469621 2577 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="59c9942842595a1405b327adb0eddb434af665e9de26dfdd45b0dcf00a583236" exitCode=0 Oct 30 19:47:02.469759 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.469696 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerDied","Data":"59c9942842595a1405b327adb0eddb434af665e9de26dfdd45b0dcf00a583236"} Oct 30 19:47:02.471378 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.471330 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" event={"ID":"dc3812fa-69b5-4e6e-bca9-91a3b191ed32","Type":"ContainerStarted","Data":"e027c0fe90163ef2f1e625292c199c0a9201aa4d3a843bfcd8dddc6cd9424b65"} Oct 30 19:47:02.513362 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.513317 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" podStartSLOduration=4.377488783 podStartE2EDuration="12.513304927s" totalImagesPullingTime="8.135816144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:01.600535114 +0000 UTC m=+11.760716324" watchObservedRunningTime="2026-10-30 19:47:02.513304927 +0000 UTC m=+12.673486121" Oct 30 19:47:02.513821 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.513790 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9tnc4" podStartSLOduration=4.897968287 podStartE2EDuration="12.513781533s" totalImagesPullingTime="7.615813246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:02.513480954 +0000 UTC m=+12.673662152" watchObservedRunningTime="2026-10-30 19:47:02.513781533 +0000 UTC m=+12.673962731" Oct 30 19:47:02.564213 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:02.564112 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" podStartSLOduration=3.07866064 podStartE2EDuration="12.564097523s" totalImagesPullingTime="9.485436883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:02.563566221 +0000 UTC m=+12.723747417" watchObservedRunningTime="2026-10-30 19:47:02.564097523 +0000 UTC m=+12.724278721" Oct 30 19:47:03.476113 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:03.476082 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerStarted","Data":"005a477cd8cc9fefcaf2305e84d74c932acfd9e09c0c9fba24fc0c5a07fed6f6"} Oct 30 19:47:04.294279 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:04.294247 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:04.294441 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:04.294255 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:04.294441 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:04.294405 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:04.294583 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:04.294452 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:04.294583 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:04.294539 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:04.294648 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:04.294610 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:04.481479 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:04.481442 2577 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="005a477cd8cc9fefcaf2305e84d74c932acfd9e09c0c9fba24fc0c5a07fed6f6" exitCode=0 Oct 30 19:47:04.482013 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:04.481492 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerDied","Data":"005a477cd8cc9fefcaf2305e84d74c932acfd9e09c0c9fba24fc0c5a07fed6f6"} Oct 30 19:47:06.297661 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:06.297632 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:06.298270 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:06.297735 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:06.298270 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:06.297747 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:06.298270 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:06.297854 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:06.298270 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:06.297887 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:06.298270 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:06.297948 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:07.489742 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:07.489521 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"08583052a04d61f55c563e04148eebdfa33361c8739635277cedf61d51a8ee57"} Oct 30 19:47:07.490120 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:07.489829 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"1d409e4e922b5d9b59acaff382d8d99f1dc54ff36c3fd50bb535e3d1950e2b70"} Oct 30 19:47:07.490120 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:07.489874 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"f53106f64912a1191d8240ee5781a5b0f44778ab3b791136d97fbefb40c0602f"} Oct 30 19:47:07.490120 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:07.489927 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"7c86f3641e85592be1f8c5b0339d21dbde86afc667f1c35cc193d351b12e6724"} Oct 30 19:47:07.492402 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:07.492380 2577 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="c71c60203c2803419b7a8b72d087e947d0b64ae545eca047f0e1caed485d257a" exitCode=0 Oct 30 19:47:07.492475 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:07.492420 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerDied","Data":"c71c60203c2803419b7a8b72d087e947d0b64ae545eca047f0e1caed485d257a"} Oct 30 19:47:08.050336 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.050120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:08.050488 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.050357 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:08.050488 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.050262 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:47:08.050488 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.050463 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret podName:a3757b61-1b91-4739-a7fc-ecbfcb4feea6 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:24.050443611 +0000 UTC m=+34.210624792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret") pod "global-pull-secret-syncer-jvsfx" (UID: "a3757b61-1b91-4739-a7fc-ecbfcb4feea6") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:47:08.050488 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.050465 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:47:08.050668 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.050499 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs podName:62561598-09a8-4cb2-b04d-81ce7c0b8411 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:24.05048929 +0000 UTC m=+34.210670480 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs") pod "network-metrics-daemon-lb8sw" (UID: "62561598-09a8-4cb2-b04d-81ce7c0b8411") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:47:08.151344 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.151303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:08.151501 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.151432 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:47:08.151501 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.151447 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:47:08.151501 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.151457 2577 projected.go:196] Error preparing data for projected volume kube-api-access-wdmfq for pod openshift-network-diagnostics/network-check-target-bvdgb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:47:08.151501 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.151499 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq podName:da64e35d-dc35-4bc8-89cb-23a7eaf75836 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:24.151486205 +0000 UTC m=+34.311667380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdmfq" (UniqueName: "kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq") pod "network-check-target-bvdgb" (UID: "da64e35d-dc35-4bc8-89cb-23a7eaf75836") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:47:08.294488 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.294456 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:08.294639 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.294510 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:08.294639 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.294528 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:08.294639 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.294592 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:08.294779 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.294676 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:08.294779 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:08.294737 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:08.501019 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.500981 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"ce14ed3560291d1a7253601e4e4ea106630636ce70a01d8f1e09a31cf3e9685e"} Oct 30 19:47:08.501453 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.501092 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"d4ba22ff89d29c7bcfc4a672def7995f6eea8f2b6fbfd4924309f71698ba961b"} Oct 30 19:47:08.503942 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.503910 2577 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="4eaa7f25ed27b0beef4e2d0783eac63ede6069d11382f3c6253ed953d5719ebb" exitCode=0 Oct 30 19:47:08.504032 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:08.503956 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerDied","Data":"4eaa7f25ed27b0beef4e2d0783eac63ede6069d11382f3c6253ed953d5719ebb"} Oct 30 19:47:09.509416 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:09.509374 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" event={"ID":"298e0d65-779b-4fd0-8931-43596a15d5fa","Type":"ContainerStarted","Data":"d71c7ef166d09ba6b22b18bb507a1e6500be0bb70ab0422eab1e2c72f08bab6d"} Oct 30 19:47:10.295684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:10.295654 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:10.295840 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:10.295739 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:10.295910 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:10.295841 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:10.295983 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:10.295964 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:10.296041 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:10.296026 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:10.296176 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:10.296128 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:10.514201 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:10.514168 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"e84dfbd4029a9101c221656efe5255888a07ceb5a5c041dabf376ca109c29c02"} Oct 30 19:47:11.956656 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:11.956614 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:12.130008 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.129984 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:12.294614 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.294581 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:12.294806 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.294716 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:12.294806 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:12.294727 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:12.294806 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:12.294788 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:12.294969 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.294809 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:12.295027 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:12.294982 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:12.521668 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.521581 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" event={"ID":"e1e0c2b5-3116-4963-b1df-fef06ea58937","Type":"ContainerStarted","Data":"e47fb9f181277b7ee1989772492f9a308b13adf7d7896c0850cf69c8c83d630a"} Oct 30 19:47:12.521939 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.521920 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:47:12.537550 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.537528 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:47:12.619339 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.619289 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gnmqz" podStartSLOduration=11.833614932 podStartE2EDuration="22.619276675s" totalImagesPullingTime="10.785661743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:09.587651361 +0000 UTC m=+19.747832559" watchObservedRunningTime="2026-10-30 19:47:12.619276675 +0000 UTC m=+22.779457873" Oct 30 19:47:12.619721 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:12.619697 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" podStartSLOduration=8.072014493 podStartE2EDuration="22.619689074s" totalImagesPullingTime="14.547674581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:12.605550849 +0000 UTC m=+22.765732045" watchObservedRunningTime="2026-10-30 19:47:12.619689074 +0000 UTC m=+22.779870305" Oct 30 19:47:13.524777 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.524744 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:47:13.525444 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.525421 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:47:13.541608 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.541584 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:47:13.906009 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.905634 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jvsfx"] Oct 30 19:47:13.906195 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.906137 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:13.906361 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:13.906325 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:13.907270 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.907235 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bvdgb"] Oct 30 19:47:13.907370 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.907331 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:13.907491 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:13.907470 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:13.907998 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.907966 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lb8sw"] Oct 30 19:47:13.908097 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:13.908084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:13.908219 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:13.908197 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:14.526340 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:14.526311 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:47:14.654053 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:14.654017 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:14.792450 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:14.792358 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:14.959332 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:14.959301 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:15.079958 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:15.079858 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:15.294484 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:15.294457 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:15.294625 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:15.294463 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:15.294625 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:15.294548 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:15.294737 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:15.294464 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:15.294737 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:15.294636 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:15.294737 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:15.294718 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:15.528429 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:15.528402 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:47:15.791467 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:15.791381 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:15.924403 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:15.924375 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:16.419154 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:16.419119 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:17.295166 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:17.295125 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:17.295166 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:17.295143 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:17.295737 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:17.295237 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:17.295737 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:17.295130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:17.295737 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:17.295352 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:17.295737 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:17.295437 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:17.467154 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:17.467108 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:17.664994 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:17.664966 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:17.820851 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:17.820797 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:18.023629 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:18.023555 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:18.192003 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:18.191976 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:18.335693 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:18.335629 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p6pc9_bb90a295-9822-4874-9014-0a91ac7a73c5/dns-node-resolver/0.log" Oct 30 19:47:18.633015 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:18.632991 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:18.980684 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:18.980614 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-44glf" Oct 30 19:47:18.980833 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:18.980783 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:47:18.981195 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:18.981181 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-44glf" Oct 30 19:47:19.136283 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.136253 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ghl6s_bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57/node-ca/0.log" Oct 30 19:47:19.137128 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.137108 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:19.294689 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.294629 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:19.294689 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.294656 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:19.294877 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:19.294728 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:19.294877 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.294764 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:19.294877 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:19.294828 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:19.294995 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:19.294911 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:19.472872 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.472834 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:19.601273 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.601193 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:47:19.601431 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.601402 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:47:19.613273 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.613249 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:47:19.630147 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.630125 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:19.756273 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.756241 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:19.925126 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:19.925102 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:20.847320 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:20.847293 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:20.992034 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:20.992001 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:21.134544 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.134517 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:21.277697 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.277670 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:21.294271 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.294249 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:21.294271 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.294264 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:21.294413 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:21.294335 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:21.294413 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.294387 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:21.294476 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:21.294454 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:21.294527 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:21.294511 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:21.420739 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.420663 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:21.563487 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.563464 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:21.749948 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.749846 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:21.923759 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:21.923731 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:22.449373 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:22.449346 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:22.574295 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:22.574268 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:23.129179 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.129136 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:23.294449 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.294422 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:23.294610 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.294422 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:23.294610 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:23.294520 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:23.294610 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.294427 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:23.294610 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:23.294599 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:23.294815 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:23.294651 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:23.387211 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.387185 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:23.513085 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.513058 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:23.648357 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.648300 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:23.853175 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.853150 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:23.980950 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:23.980847 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:24.055439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:24.055404 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:24.055439 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:24.055441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:24.055629 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:24.055546 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:47:24.055629 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:24.055548 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:47:24.055629 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:24.055597 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs podName:62561598-09a8-4cb2-b04d-81ce7c0b8411 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:56.055584088 +0000 UTC m=+66.215765277 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs") pod "network-metrics-daemon-lb8sw" (UID: "62561598-09a8-4cb2-b04d-81ce7c0b8411") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:47:24.055629 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:24.055609 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret podName:a3757b61-1b91-4739-a7fc-ecbfcb4feea6 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:56.055603645 +0000 UTC m=+66.215784823 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret") pod "global-pull-secret-syncer-jvsfx" (UID: "a3757b61-1b91-4739-a7fc-ecbfcb4feea6") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:47:24.156557 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:24.156502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:24.156975 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:24.156677 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:47:24.156975 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:24.156695 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:47:24.156975 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:24.156704 2577 projected.go:196] Error preparing data for projected volume kube-api-access-wdmfq for pod openshift-network-diagnostics/network-check-target-bvdgb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:47:24.156975 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:24.156755 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq podName:da64e35d-dc35-4bc8-89cb-23a7eaf75836 nodeName:}" failed. No retries permitted until 2026-10-30 19:47:56.156743314 +0000 UTC m=+66.316924489 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wdmfq" (UniqueName: "kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq") pod "network-check-target-bvdgb" (UID: "da64e35d-dc35-4bc8-89cb-23a7eaf75836") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:47:25.159131 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:25.159099 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:25.294843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:25.294813 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:25.294843 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:25.294839 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:25.295105 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:25.294964 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:25.295105 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:25.294998 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:25.295105 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:25.295035 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:25.295262 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:25.295114 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:25.331050 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:25.331020 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:25.540219 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:25.540151 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:25.743033 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:25.743000 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:25.943486 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:25.943450 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:26.926947 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:26.926919 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:27.060413 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.060387 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:27.294400 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.294326 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:27.294400 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.294351 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:27.294569 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.294326 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:27.294569 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:27.294430 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:27.294569 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:27.294525 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:27.294671 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:27.294592 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:27.330491 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.330466 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:27.465589 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.465563 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:27.590552 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.590489 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:27.733380 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.733352 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:27.926455 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:27.926426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:28.532708 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:28.532679 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:28.728447 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:28.728412 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:29.294848 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:29.294818 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:29.294848 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:29.294848 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:29.295091 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:29.295005 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:29.295091 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:29.295054 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:29.295198 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:29.295118 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:29.295252 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:29.295208 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:30.869875 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:30.869839 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:31.294580 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:31.294553 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:31.294580 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:31.294585 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:31.294760 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:31.294586 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:31.294760 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:31.294663 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:31.294760 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:31.294732 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:31.294851 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:31.294767 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:32.881544 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:32.881510 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:33.009228 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:33.009201 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:33.295003 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:33.294925 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:33.295003 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:33.294935 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:33.295003 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:33.294955 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:33.295235 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:33.295045 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:33.295235 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:33.295105 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:33.295235 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:33.295162 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:35.294629 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:35.294601 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:35.295121 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:35.294639 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:35.295121 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:35.294703 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:35.295121 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:35.294725 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:35.295121 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:35.294801 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:35.295121 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:35.294855 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:36.937672 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:36.937521 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:37.294296 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.294216 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:37.294296 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.294250 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:37.294488 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:37.294300 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvdgb" podUID="da64e35d-dc35-4bc8-89cb-23a7eaf75836" Oct 30 19:47:37.294488 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.294352 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:37.294488 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:37.294453 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvsfx" podUID="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" Oct 30 19:47:37.294598 ip-10-0-142-168 kubenswrapper[2577]: E1030 19:47:37.294499 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lb8sw" podUID="62561598-09a8-4cb2-b04d-81ce7c0b8411" Oct 30 19:47:37.585357 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.585291 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeReady" Oct 30 19:47:37.585483 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.585388 2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 19:47:37.657042 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.657022 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cwb2s"] Oct 30 19:47:37.659579 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.659477 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.659887 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.659864 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f8s4w"] Oct 30 19:47:37.662399 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.662382 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:47:37.677987 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.677946 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 19:47:37.678073 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.678008 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 19:47:37.678073 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.678048 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l5g68\"" Oct 30 19:47:37.683067 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.683049 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-887vm"] Oct 30 19:47:37.685475 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.685456 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.686226 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.686208 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f8s4w"] Oct 30 19:47:37.689521 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.689503 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cwb2s"] Oct 30 19:47:37.709844 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.709825 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-887vm"] Oct 30 19:47:37.763134 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.763110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65d4fb96-8b10-487d-9912-bd71dd507120-cert\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:47:37.763219 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.763145 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46a18c80-179d-46ba-85aa-1489814be80e-config-volume\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.763219 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.763165 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46a18c80-179d-46ba-85aa-1489814be80e-tmp-dir\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.763219 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.763189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g5bf\" (UniqueName: \"kubernetes.io/projected/46a18c80-179d-46ba-85aa-1489814be80e-kube-api-access-2g5bf\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.763312 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.763274 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46a18c80-179d-46ba-85aa-1489814be80e-metrics-tls\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.763347 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.763321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7qdn\" (UniqueName: \"kubernetes.io/projected/65d4fb96-8b10-487d-9912-bd71dd507120-kube-api-access-c7qdn\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:47:37.777940 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.777923 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 19:47:37.778186 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.778149 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7g4j\"" Oct 30 19:47:37.778675 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.778645 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 19:47:37.778760 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.778706 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 19:47:37.778943 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.778891 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 19:47:37.779236 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.779145 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 19:47:37.779236 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.779171 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 19:47:37.779236 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.779212 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 19:47:37.781253 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.779520 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-h5z6r\"" Oct 30 19:47:37.863791 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.863734 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27586708-df16-4c22-9b2c-eb87fc94595d-data-volume\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.863791 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.863763 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdvhz\" (UniqueName: \"kubernetes.io/projected/27586708-df16-4c22-9b2c-eb87fc94595d-kube-api-access-sdvhz\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.863950 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.863798 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27586708-df16-4c22-9b2c-eb87fc94595d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.863950 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.863891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7qdn\" (UniqueName: \"kubernetes.io/projected/65d4fb96-8b10-487d-9912-bd71dd507120-kube-api-access-c7qdn\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:47:37.863950 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.863938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27586708-df16-4c22-9b2c-eb87fc94595d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.864038 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.863964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65d4fb96-8b10-487d-9912-bd71dd507120-cert\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:47:37.864038 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.864011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46a18c80-179d-46ba-85aa-1489814be80e-config-volume\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.864038 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.864029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46a18c80-179d-46ba-85aa-1489814be80e-tmp-dir\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.864210 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.864044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g5bf\" (UniqueName: \"kubernetes.io/projected/46a18c80-179d-46ba-85aa-1489814be80e-kube-api-access-2g5bf\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.864210 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.864071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46a18c80-179d-46ba-85aa-1489814be80e-metrics-tls\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.864210 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.864093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27586708-df16-4c22-9b2c-eb87fc94595d-crio-socket\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.864450 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.864430 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46a18c80-179d-46ba-85aa-1489814be80e-tmp-dir\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.864680 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.864659 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46a18c80-179d-46ba-85aa-1489814be80e-config-volume\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.867507 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.867485 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65d4fb96-8b10-487d-9912-bd71dd507120-cert\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:47:37.867581 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.867503 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46a18c80-179d-46ba-85aa-1489814be80e-metrics-tls\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.893565 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.893545 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7qdn\" (UniqueName: \"kubernetes.io/projected/65d4fb96-8b10-487d-9912-bd71dd507120-kube-api-access-c7qdn\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:47:37.893658 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.893598 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g5bf\" (UniqueName: \"kubernetes.io/projected/46a18c80-179d-46ba-85aa-1489814be80e-kube-api-access-2g5bf\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.915806 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.915788 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:37.964438 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.964416 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27586708-df16-4c22-9b2c-eb87fc94595d-crio-socket\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.964728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.964449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27586708-df16-4c22-9b2c-eb87fc94595d-data-volume\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.964728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.964464 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdvhz\" (UniqueName: \"kubernetes.io/projected/27586708-df16-4c22-9b2c-eb87fc94595d-kube-api-access-sdvhz\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.964728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.964614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27586708-df16-4c22-9b2c-eb87fc94595d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.964728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.964654 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27586708-df16-4c22-9b2c-eb87fc94595d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.964728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.964613 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27586708-df16-4c22-9b2c-eb87fc94595d-crio-socket\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.964728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.964716 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27586708-df16-4c22-9b2c-eb87fc94595d-data-volume\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.965178 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.965162 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27586708-df16-4c22-9b2c-eb87fc94595d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.966823 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.966801 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27586708-df16-4c22-9b2c-eb87fc94595d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:37.973730 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.973714 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:37.975839 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.975824 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:47:37.990240 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:37.990219 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdvhz\" (UniqueName: \"kubernetes.io/projected/27586708-df16-4c22-9b2c-eb87fc94595d-kube-api-access-sdvhz\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:38.001254 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.001223 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:47:38.260973 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.260948 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:38.297775 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.297758 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f8s4w"] Oct 30 19:47:38.299579 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:38.299558 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d4fb96_8b10_487d_9912_bd71dd507120.slice/crio-8498a05bd0dea1a76f393f6262913bac53039ce037a8ac8c03682da8af01858a WatchSource:0}: Error finding container 8498a05bd0dea1a76f393f6262913bac53039ce037a8ac8c03682da8af01858a: Status 404 returned error can't find the container with id 8498a05bd0dea1a76f393f6262913bac53039ce037a8ac8c03682da8af01858a Oct 30 19:47:38.308931 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.308891 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cwb2s"] Oct 30 19:47:38.311656 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.311633 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-887vm"] Oct 30 19:47:38.311786 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:38.311767 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a18c80_179d_46ba_85aa_1489814be80e.slice/crio-4348d980cbcc3ed0b92fd34a2facffc6d4a6318e67cc2425df24d79218a2dc0a WatchSource:0}: Error finding container 4348d980cbcc3ed0b92fd34a2facffc6d4a6318e67cc2425df24d79218a2dc0a: Status 404 returned error can't find the container with id 4348d980cbcc3ed0b92fd34a2facffc6d4a6318e67cc2425df24d79218a2dc0a Oct 30 19:47:38.314527 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:38.314506 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27586708_df16_4c22_9b2c_eb87fc94595d.slice/crio-c3ae80f992f83041d46b23c1ad5bd0cc8a3ce485ad9d199b83d0e12f8da4ccc4 WatchSource:0}: Error finding container c3ae80f992f83041d46b23c1ad5bd0cc8a3ce485ad9d199b83d0e12f8da4ccc4: Status 404 returned error can't find the container with id c3ae80f992f83041d46b23c1ad5bd0cc8a3ce485ad9d199b83d0e12f8da4ccc4 Oct 30 19:47:38.422555 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.422530 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:38.577242 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.577169 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-887vm" event={"ID":"27586708-df16-4c22-9b2c-eb87fc94595d","Type":"ContainerStarted","Data":"1739b94f2709d52737c158184cb75b5d1ba21aeea5adb1cdceb9f1cd2c9daeea"} Oct 30 19:47:38.577242 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.577228 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-887vm" event={"ID":"27586708-df16-4c22-9b2c-eb87fc94595d","Type":"ContainerStarted","Data":"c3ae80f992f83041d46b23c1ad5bd0cc8a3ce485ad9d199b83d0e12f8da4ccc4"} Oct 30 19:47:38.578235 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.578216 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f8s4w" event={"ID":"65d4fb96-8b10-487d-9912-bd71dd507120","Type":"ContainerStarted","Data":"8498a05bd0dea1a76f393f6262913bac53039ce037a8ac8c03682da8af01858a"} Oct 30 19:47:38.579308 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:38.579291 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwb2s" event={"ID":"46a18c80-179d-46ba-85aa-1489814be80e","Type":"ContainerStarted","Data":"4348d980cbcc3ed0b92fd34a2facffc6d4a6318e67cc2425df24d79218a2dc0a"} Oct 30 19:47:39.139766 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.139735 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:39.294702 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.294620 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:39.294702 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.294650 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:39.294702 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.294666 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:39.302237 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.302216 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:39.327193 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.327166 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 19:47:39.327193 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.327166 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wnngx\"" Oct 30 19:47:39.327367 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.327177 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 19:47:39.327367 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.327284 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2gzs4\"" Oct 30 19:47:39.327367 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.327300 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 19:47:39.327367 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.327343 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 19:47:39.327549 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.327387 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mbspd\"" Oct 30 19:47:39.430913 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.430863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:39.584779 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:39.584656 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-887vm" event={"ID":"27586708-df16-4c22-9b2c-eb87fc94595d","Type":"ContainerStarted","Data":"8ea4a9e43253f4b69de86e4c1a63553963b55d5ae8e8979ba7416dcfcb896245"} Oct 30 19:47:40.592142 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:40.591211 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwb2s" event={"ID":"46a18c80-179d-46ba-85aa-1489814be80e","Type":"ContainerStarted","Data":"7bd1b6da7f2a19adf5d38e6db11fa767299f86d405ebf0cb462bb38395668668"} Oct 30 19:47:40.592142 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:40.591275 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwb2s" event={"ID":"46a18c80-179d-46ba-85aa-1489814be80e","Type":"ContainerStarted","Data":"689ed28bbe7238394676a4ba6187d1bf71b20f40f9b63465147ac500f2ba3fcf"} Oct 30 19:47:40.592142 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:40.591325 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:40.846981 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:40.846877 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:40.906296 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:40.906242 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-cwb2s" podStartSLOduration=2.124344395 podStartE2EDuration="3.906224953s" totalImagesPullingTime="1.781880558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:40.90524027 +0000 UTC m=+51.065421478" watchObservedRunningTime="2026-10-30 19:47:40.906224953 +0000 UTC m=+51.066406151" Oct 30 19:47:40.974990 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:40.974962 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:41.065177 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.065145 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh"] Oct 30 19:47:41.068122 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.068093 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.089301 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.089273 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh"] Oct 30 19:47:41.089422 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.089377 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 30 19:47:41.091108 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.091087 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jnc65\"" Oct 30 19:47:41.092123 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.092099 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 19:47:41.092227 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.092212 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 19:47:41.092335 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.092311 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 19:47:41.132638 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.132611 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:41.189088 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.189059 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 30 19:47:41.193541 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.193519 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4w2g\" (UniqueName: \"kubernetes.io/projected/c1873229-3e23-4ff6-9f57-f8520e9d9852-kube-api-access-t4w2g\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.193615 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.193558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1873229-3e23-4ff6-9f57-f8520e9d9852-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.193677 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.193660 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.193711 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.193702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.262474 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.262441 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:41.294736 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.294696 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.294947 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.294755 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.294947 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.294802 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4w2g\" (UniqueName: \"kubernetes.io/projected/c1873229-3e23-4ff6-9f57-f8520e9d9852-kube-api-access-t4w2g\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.294947 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.294829 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1873229-3e23-4ff6-9f57-f8520e9d9852-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.295688 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.295655 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1873229-3e23-4ff6-9f57-f8520e9d9852-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.297483 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.297466 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.297546 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.297516 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.322169 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.322134 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4w2g\" (UniqueName: \"kubernetes.io/projected/c1873229-3e23-4ff6-9f57-f8520e9d9852-kube-api-access-t4w2g\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.384524 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.384432 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:47:41.595751 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.595714 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-887vm" event={"ID":"27586708-df16-4c22-9b2c-eb87fc94595d","Type":"ContainerStarted","Data":"a2aeafab1d6dd06044ece6a18945a7deee2402a83f22b13e45ee382166bbe3d8"} Oct 30 19:47:41.597301 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.597275 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f8s4w" event={"ID":"65d4fb96-8b10-487d-9912-bd71dd507120","Type":"ContainerStarted","Data":"cae3e20afecab65a79e70301b60de66854eadbaa6281dab1e3fcc00de5098ea7"} Oct 30 19:47:41.643653 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.643557 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-887vm" podStartSLOduration=2.441352426 podStartE2EDuration="4.643540817s" totalImagesPullingTime="2.202188391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:47:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:41.642951677 +0000 UTC m=+51.803132873" watchObservedRunningTime="2026-10-30 19:47:41.643540817 +0000 UTC m=+51.803722015" Oct 30 19:47:41.684223 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.684192 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh"] Oct 30 19:47:41.687083 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:41.687048 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1873229_3e23_4ff6_9f57_f8520e9d9852.slice/crio-84cf4865b87984598fe3b16ba807871a01d29dd831fb876a67a26dd750d45fa6 WatchSource:0}: Error finding container 84cf4865b87984598fe3b16ba807871a01d29dd831fb876a67a26dd750d45fa6: Status 404 returned error can't find the container with id 84cf4865b87984598fe3b16ba807871a01d29dd831fb876a67a26dd750d45fa6 Oct 30 19:47:41.687586 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:41.687546 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f8s4w" podStartSLOduration=1.851481481 podStartE2EDuration="4.687533315s" totalImagesPullingTime="2.836051834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:41.687324893 +0000 UTC m=+51.847506091" watchObservedRunningTime="2026-10-30 19:47:41.687533315 +0000 UTC m=+51.847714512" Oct 30 19:47:42.249394 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.249367 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-57b59b9797-ptgs8"] Oct 30 19:47:42.252386 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.252366 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.270770 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.270752 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 19:47:42.273094 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.273070 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 19:47:42.273182 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.273130 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rgk57\"" Oct 30 19:47:42.274721 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.274703 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 19:47:42.275018 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.274995 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 19:47:42.275018 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.275016 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 19:47:42.284580 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.284566 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 19:47:42.302624 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.302587 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 19:47:42.303878 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.303857 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57b59b9797-ptgs8"] Oct 30 19:47:42.375208 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.375181 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 19:47:42.404151 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.404118 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.404314 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.404181 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.404314 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.404248 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.404314 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.404285 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.404486 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.404441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.404549 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.404483 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.404549 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.404513 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z65z\" (UniqueName: \"kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.404632 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.404595 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505464 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505464 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z65z\" (UniqueName: \"kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505464 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505446 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505714 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505480 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505714 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505520 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505714 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505714 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505714 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505651 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.505969 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.505861 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.506435 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.506411 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.506549 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.506438 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.506910 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.506872 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.507258 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.507236 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.509086 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.509067 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.509403 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.509377 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.510219 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.510193 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:42.532454 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.532435 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z65z\" (UniqueName: \"kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.565854 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.565824 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:42.601633 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.601596 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" event={"ID":"c1873229-3e23-4ff6-9f57-f8520e9d9852","Type":"ContainerStarted","Data":"84cf4865b87984598fe3b16ba807871a01d29dd831fb876a67a26dd750d45fa6"} Oct 30 19:47:42.643592 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.643553 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:42.865537 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:42.865459 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57b59b9797-ptgs8"] Oct 30 19:47:42.869135 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:42.869110 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a9532b_3f87_4fac_816a_d293b921ce5c.slice/crio-38efa6c6146cff0df453fce2e0de42b581de3b4cb80b73e6c2279da08ddeec2d WatchSource:0}: Error finding container 38efa6c6146cff0df453fce2e0de42b581de3b4cb80b73e6c2279da08ddeec2d: Status 404 returned error can't find the container with id 38efa6c6146cff0df453fce2e0de42b581de3b4cb80b73e6c2279da08ddeec2d Oct 30 19:47:43.037216 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:43.037196 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwb2s_46a18c80-179d-46ba-85aa-1489814be80e/dns/0.log" Oct 30 19:47:43.237467 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:43.237445 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwb2s_46a18c80-179d-46ba-85aa-1489814be80e/kube-rbac-proxy/0.log" Oct 30 19:47:43.609891 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:43.609806 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b59b9797-ptgs8" event={"ID":"42a9532b-3f87-4fac-816a-d293b921ce5c","Type":"ContainerStarted","Data":"38efa6c6146cff0df453fce2e0de42b581de3b4cb80b73e6c2279da08ddeec2d"} Oct 30 19:47:43.611909 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:43.611861 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" event={"ID":"c1873229-3e23-4ff6-9f57-f8520e9d9852","Type":"ContainerStarted","Data":"385adf2adb52eb13fbd62a6c988468056eae4eb50fc6046b87cabc286b006901"} Oct 30 19:47:43.612031 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:43.611930 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" event={"ID":"c1873229-3e23-4ff6-9f57-f8520e9d9852","Type":"ContainerStarted","Data":"a851377a6e7c11d33a6064d633a6d75f18478086f9666a043223b4e0675043df"} Oct 30 19:47:43.660443 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:43.660405 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" podStartSLOduration=1.4183596330000001 podStartE2EDuration="2.660392851s" totalImagesPullingTime="1.242033218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:43.658050285 +0000 UTC m=+53.818231481" watchObservedRunningTime="2026-10-30 19:47:43.660392851 +0000 UTC m=+53.820574047" Oct 30 19:47:43.853793 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:43.853762 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:44.238554 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:44.238525 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p6pc9_bb90a295-9822-4874-9014-0a91ac7a73c5/dns-node-resolver/0.log" Oct 30 19:47:44.648762 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:44.648728 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:44.794235 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:44.794206 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:45.244523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.244498 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ghl6s_bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57/node-ca/0.log" Oct 30 19:47:45.575153 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.575084 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:45.618819 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.618791 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b59b9797-ptgs8" event={"ID":"42a9532b-3f87-4fac-816a-d293b921ce5c","Type":"ContainerStarted","Data":"59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913"} Oct 30 19:47:45.680767 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.680720 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-57b59b9797-ptgs8" podStartSLOduration=1.140652539 podStartE2EDuration="3.680707041s" totalImagesPullingTime="2.540054502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:45.66710814 +0000 UTC m=+55.827289337" watchObservedRunningTime="2026-10-30 19:47:45.680707041 +0000 UTC m=+55.840888294" Oct 30 19:47:45.709218 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.709197 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:45.821193 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.821167 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dcpkx"] Oct 30 19:47:45.824328 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.824309 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830042 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.829982 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830042 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.830020 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-root\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830166 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.830054 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830166 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.830113 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-sys\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830166 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.830143 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-metrics-client-ca\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830166 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.830159 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-textfile\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830411 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.830178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-tls\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830411 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.830308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-wtmp\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.830411 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.830344 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdxr6\" (UniqueName: \"kubernetes.io/projected/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-kube-api-access-qdxr6\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.837664 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.837648 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f8s4w_65d4fb96-8b10-487d-9912-bd71dd507120/serve-healthcheck-canary/0.log" Oct 30 19:47:45.875168 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.875145 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:45.889715 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.889696 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 19:47:45.897863 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.897841 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 19:47:45.897966 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.897869 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-67dw4\"" Oct 30 19:47:45.898008 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.897975 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 19:47:45.930805 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930783 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-wtmp\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.930886 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdxr6\" (UniqueName: \"kubernetes.io/projected/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-kube-api-access-qdxr6\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.930886 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.930886 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-root\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930914 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-sys\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930951 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-metrics-client-ca\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930970 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-textfile\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930977 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-root\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-tls\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.930978 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-wtmp\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.931024 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-sys\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931370 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.931257 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-textfile\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931711 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.931616 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.931711 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.931627 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-metrics-client-ca\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.933339 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.933320 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-tls\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.933414 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.933406 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:45.966548 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:45.966528 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdxr6\" (UniqueName: \"kubernetes.io/projected/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-kube-api-access-qdxr6\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:46.047055 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:46.047023 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:46.138871 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:46.138845 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:47:46.150495 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:46.150473 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba49ca5_7b3e_42bc_aa61_59004aeb73dc.slice/crio-f2f79ed46b081f577502da86affe4b5f13e7c71a6535249600fba0757a9204e2 WatchSource:0}: Error finding container f2f79ed46b081f577502da86affe4b5f13e7c71a6535249600fba0757a9204e2: Status 404 returned error can't find the container with id f2f79ed46b081f577502da86affe4b5f13e7c71a6535249600fba0757a9204e2 Oct 30 19:47:46.275602 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:46.275573 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:46.425310 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:46.425244 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:46.623531 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:46.623484 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dcpkx" event={"ID":"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc","Type":"ContainerStarted","Data":"f2f79ed46b081f577502da86affe4b5f13e7c71a6535249600fba0757a9204e2"} Oct 30 19:47:47.031404 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.031379 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:47.190498 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.190471 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:47.368128 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.368059 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 19:47:47.371437 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.371421 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.390932 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.390887 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 19:47:47.396445 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.396427 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:47.403809 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.403768 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 19:47:47.403927 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.403866 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 19:47:47.403996 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.403929 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 19:47:47.403996 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.403944 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 19:47:47.403996 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.403956 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 19:47:47.403996 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.403977 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 19:47:47.404116 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.404016 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 19:47:47.404116 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.404054 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 19:47:47.404176 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.404114 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2llhk\"" Oct 30 19:47:47.420531 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.420514 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 19:47:47.444790 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.444769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.444886 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.444856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.444886 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.444879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.444992 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.444955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.444992 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.444976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.445059 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.444995 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.445059 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.445023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.445059 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.445038 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.445162 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.445093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.445162 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.445135 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmc7x\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.445241 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.445224 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.445287 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.445274 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.445330 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.445316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546487 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546598 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546494 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546598 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546598 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546549 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546598 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546567 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546598 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546838 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546605 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546838 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546633 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546838 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546673 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546838 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546697 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546838 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546724 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546838 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546750 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmc7x\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.546838 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.546790 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.548055 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.547432 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.548055 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.547823 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.548055 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.548014 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.549708 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.549682 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.550218 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.550180 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.550356 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.550311 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.550656 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.550632 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.550773 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.550753 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.550829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.550797 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.550829 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.550804 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.550950 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.550936 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.551398 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.551383 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.574233 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.574214 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmc7x\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:47.628455 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.628400 2577 generic.go:356] "Generic (PLEG): container finished" podID="0ba49ca5-7b3e-42bc-aa61-59004aeb73dc" containerID="9105012d760598cccaab9c757aa6fbc0a1b7f3a89fabbfc9f0c180cf21fdfeda" exitCode=0 Oct 30 19:47:47.628455 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.628438 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dcpkx" event={"ID":"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc","Type":"ContainerDied","Data":"9105012d760598cccaab9c757aa6fbc0a1b7f3a89fabbfc9f0c180cf21fdfeda"} Oct 30 19:47:47.695265 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:47.695246 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:47:48.060644 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:48.060621 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 19:47:48.062522 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:48.062494 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f5f63c_a55a_4f7c_858c_bb3051ef73be.slice/crio-d03e243eafcb3ce5236c92e0eb4fe70065a94c06f91e1cba3ea5b8be5a092566 WatchSource:0}: Error finding container d03e243eafcb3ce5236c92e0eb4fe70065a94c06f91e1cba3ea5b8be5a092566: Status 404 returned error can't find the container with id d03e243eafcb3ce5236c92e0eb4fe70065a94c06f91e1cba3ea5b8be5a092566 Oct 30 19:47:48.634052 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:48.634014 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerStarted","Data":"d03e243eafcb3ce5236c92e0eb4fe70065a94c06f91e1cba3ea5b8be5a092566"} Oct 30 19:47:48.636432 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:48.636405 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dcpkx" event={"ID":"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc","Type":"ContainerStarted","Data":"f5c57a0af7daac5db59992ace6bb3cf555eac63cd407f28471bc65c7de150919"} Oct 30 19:47:48.636581 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:48.636455 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dcpkx" event={"ID":"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc","Type":"ContainerStarted","Data":"456eea299737597822781640b94ceface7ae0f6c50223d9017b5896eeea2e352"} Oct 30 19:47:48.687252 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:48.687209 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dcpkx" podStartSLOduration=2.779895546 podStartE2EDuration="3.68719729s" totalImagesPullingTime="833.247506ms" totalInitContainerRuntime="74.054238ms" isStatefulPod=true podCreationTimestamp="2026-10-30 19:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:48.686320853 +0000 UTC m=+58.846502061" watchObservedRunningTime="2026-10-30 19:47:48.68719729 +0000 UTC m=+58.847378487" Oct 30 19:47:49.204539 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:49.204498 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:49.340630 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:49.340549 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:49.481164 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:49.481132 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:49.627540 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:49.627466 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:49.640399 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:49.640367 2577 generic.go:356] "Generic (PLEG): container finished" podID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerID="9503fd4b4d23f77269e7bdb0f71b093fe36d61208fc9c169139ba6cd9d316a2c" exitCode=0 Oct 30 19:47:49.640577 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:49.640455 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerDied","Data":"9503fd4b4d23f77269e7bdb0f71b093fe36d61208fc9c169139ba6cd9d316a2c"} Oct 30 19:47:50.401949 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.401915 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w"] Oct 30 19:47:50.406420 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.406397 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.437213 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.437175 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 19:47:50.437213 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.437199 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 19:47:50.437409 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.437227 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 19:47:50.437409 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.437254 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 19:47:50.437409 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.437291 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-25nmp\"" Oct 30 19:47:50.438235 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.438212 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7krm2j7bnghi9\"" Oct 30 19:47:50.452167 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.452137 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w"] Oct 30 19:47:50.475089 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.475056 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-client-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.475089 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.475092 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8lff\" (UniqueName: \"kubernetes.io/projected/a6f79c4a-177a-4512-b817-92404fd99b41-kube-api-access-n8lff\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.475362 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.475163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.475362 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.475188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-client-certs\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.475362 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.475327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-metrics-server-audit-profiles\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.475509 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.475388 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-tls\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.475509 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.475441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a6f79c4a-177a-4512-b817-92404fd99b41-audit-log\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.576543 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.576498 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-metrics-server-audit-profiles\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.576732 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.576585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-tls\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.576732 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.576614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a6f79c4a-177a-4512-b817-92404fd99b41-audit-log\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.576732 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.576686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-client-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.576732 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.576715 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8lff\" (UniqueName: \"kubernetes.io/projected/a6f79c4a-177a-4512-b817-92404fd99b41-kube-api-access-n8lff\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.576990 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.576795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.576990 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.576824 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-client-certs\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.577118 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.577093 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a6f79c4a-177a-4512-b817-92404fd99b41-audit-log\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.577708 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.577683 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.577863 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.577786 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-metrics-server-audit-profiles\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.579551 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.579530 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-client-certs\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.580050 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.580024 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-client-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.580125 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.580108 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-tls\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.599801 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.599768 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cwb2s" Oct 30 19:47:50.606048 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.606025 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8lff\" (UniqueName: \"kubernetes.io/projected/a6f79c4a-177a-4512-b817-92404fd99b41-kube-api-access-n8lff\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.629816 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.629783 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:50.719571 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.719497 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:47:50.877649 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:50.877618 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:51.040971 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:51.040948 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w"] Oct 30 19:47:51.044080 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:51.044057 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f79c4a_177a_4512_b817_92404fd99b41.slice/crio-272abb5096c53f8c612c69d6d4490663181c3e3ac2359d3f89490ec57116fb67 WatchSource:0}: Error finding container 272abb5096c53f8c612c69d6d4490663181c3e3ac2359d3f89490ec57116fb67: Status 404 returned error can't find the container with id 272abb5096c53f8c612c69d6d4490663181c3e3ac2359d3f89490ec57116fb67 Oct 30 19:47:51.651832 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:51.651797 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerStarted","Data":"2bc3842b2b8607e5e6a013304dc5d5c3908ce58fe0fb6a2406362be2c0b083d7"} Oct 30 19:47:51.652358 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:51.652016 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerStarted","Data":"84c9967ee25c8ae09bc22c3cc5c5ada991268f9dad75ff9579438c44d04e0e37"} Oct 30 19:47:51.652358 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:51.652085 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerStarted","Data":"5d66f1b4008772adaa5200c686c02bb67b02af6469da3cc7515f1e812e9b15a0"} Oct 30 19:47:51.652358 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:51.652140 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerStarted","Data":"7eae5572e68c0c3cfeb00f38f811c3628fa3047d236e0a2a4dcebee06fe927af"} Oct 30 19:47:51.652358 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:51.652178 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerStarted","Data":"1bd19a6f609bc25020a3c33285b08cf9d49013f0527c5dcdbf85bfa99f863959"} Oct 30 19:47:51.657471 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:51.657444 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" event={"ID":"a6f79c4a-177a-4512-b817-92404fd99b41","Type":"ContainerStarted","Data":"272abb5096c53f8c612c69d6d4490663181c3e3ac2359d3f89490ec57116fb67"} Oct 30 19:47:52.037014 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.036940 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:52.566649 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.566580 2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:52.566649 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.566636 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:52.571567 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.571545 2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:52.664379 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.664344 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerStarted","Data":"f0063b501afc10b0774b76a383481b02f0dd79d21917143cb186dbb1b0ea2785"} Oct 30 19:47:52.665855 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.665831 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" event={"ID":"a6f79c4a-177a-4512-b817-92404fd99b41","Type":"ContainerStarted","Data":"5c698a52954539e51d4b4f9296828b401c5a8c26feb1a0ad3192dbc2756f580c"} Oct 30 19:47:52.669529 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.669511 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:47:52.744956 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.744909 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.680051765 podStartE2EDuration="5.744878506s" totalImagesPullingTime="3.055395327s" totalInitContainerRuntime="9.431414ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:52.743256064 +0000 UTC m=+62.903437291" watchObservedRunningTime="2026-10-30 19:47:52.744878506 +0000 UTC m=+62.905059705" Oct 30 19:47:52.853304 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.853219 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" podStartSLOduration=1.60762749 podStartE2EDuration="2.853205093s" totalImagesPullingTime="1.245577603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:52.852146971 +0000 UTC m=+63.012328168" watchObservedRunningTime="2026-10-30 19:47:52.853205093 +0000 UTC m=+63.013386290" Oct 30 19:47:52.971017 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.970988 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:47:52.974781 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.974765 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:52.995364 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.995340 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 19:47:52.997966 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.997946 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 19:47:52.997966 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.997958 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k6b6j\"" Oct 30 19:47:52.998157 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.997953 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 19:47:52.998157 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.998031 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 19:47:52.999888 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.999850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:52.999888 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.999878 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 19:47:53.000025 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.999876 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 19:47:53.000025 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.999911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000025 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:52.999981 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000025 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000000 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000183 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000183 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000183 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000183 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000183 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000173 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzrnv\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000368 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000201 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 19:47:53.000368 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000368 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000284 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000368 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000362 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000489 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000382 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000489 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000448 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000489 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000467 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000572 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000491 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000572 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000522 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000640 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000640 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.000640 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.000607 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.002844 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.002821 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:53.013756 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.013713 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9uipjplehrdud\"" Oct 30 19:47:53.013953 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.013754 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 19:47:53.013953 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.013772 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 19:47:53.013953 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.013782 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 19:47:53.013953 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.013776 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 19:47:53.014114 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.013958 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 19:47:53.029268 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.027393 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 19:47:53.034002 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.033977 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 19:47:53.039309 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.039286 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:47:53.101992 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.101964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102119 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102014 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102119 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102119 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102119 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102063 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102119 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102119 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102123 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102147 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzrnv\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102191 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102228 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102260 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102279 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102324 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102364 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102393 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102391 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102870 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102870 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.102870 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.102470 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.104917 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.103344 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.104917 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.103872 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.104917 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.103993 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.104917 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.104084 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.104917 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.104678 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.105699 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.105466 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.107480 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.107413 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.108015 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.107986 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.108728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.108279 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.108728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.108332 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.108728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.108453 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.108728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.108526 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.108728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.108612 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.108728 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.108677 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.109002 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.108753 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.109002 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.108855 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.109071 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.109019 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.109147 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.109131 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.109373 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.109354 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.130148 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.130130 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzrnv\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.197683 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.197653 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:53.299450 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.299424 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:47:53.649336 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.649313 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:47:53.664332 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:53.664297 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a61eb1f_e59d_4050_b108_20b6408b819a.slice/crio-b932d03caaef8b6a30aba82ea26989b0a0e4f4c7849e38bfe5671eb540886c72 WatchSource:0}: Error finding container b932d03caaef8b6a30aba82ea26989b0a0e4f4c7849e38bfe5671eb540886c72: Status 404 returned error can't find the container with id b932d03caaef8b6a30aba82ea26989b0a0e4f4c7849e38bfe5671eb540886c72 Oct 30 19:47:53.669839 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.669773 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerStarted","Data":"b932d03caaef8b6a30aba82ea26989b0a0e4f4c7849e38bfe5671eb540886c72"} Oct 30 19:47:53.898330 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.898302 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f66d775b6-dq96j"] Oct 30 19:47:53.901459 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:53.901420 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.010316 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.010294 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.010467 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.010359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.010467 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.010431 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.010551 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.010479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.010551 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.010528 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glwp6\" (UniqueName: \"kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.010551 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.010546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.010678 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.010570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.010729 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.010678 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.037376 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.037351 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f66d775b6-dq96j"] Oct 30 19:47:54.111474 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.111451 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.111574 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.111487 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.111574 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.111526 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.111574 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.111543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.111574 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.111563 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.111741 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.111614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glwp6\" (UniqueName: \"kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.111741 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.111641 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.111741 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.111685 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.112039 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.112015 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.112308 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.112280 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.112367 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.112280 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.112978 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.112963 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.113105 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.113084 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.114097 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.114081 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.114324 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.114308 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.141578 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.141549 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glwp6\" (UniqueName: \"kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.211844 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.211765 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:47:54.512765 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.512540 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f66d775b6-dq96j"] Oct 30 19:47:54.514908 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:54.514873 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb569a9c0_5370_4ceb_9a82_90dad326ffad.slice/crio-5bbe69d136aefa4fd0c16be97fc6a022036f7086640c1a70cbcf76a8bc98f030 WatchSource:0}: Error finding container 5bbe69d136aefa4fd0c16be97fc6a022036f7086640c1a70cbcf76a8bc98f030: Status 404 returned error can't find the container with id 5bbe69d136aefa4fd0c16be97fc6a022036f7086640c1a70cbcf76a8bc98f030 Oct 30 19:47:54.674136 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.674102 2577 generic.go:356] "Generic (PLEG): container finished" podID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" exitCode=0 Oct 30 19:47:54.674570 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.674187 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerDied","Data":"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7"} Oct 30 19:47:54.675950 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.675927 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f66d775b6-dq96j" event={"ID":"b569a9c0-5370-4ceb-9a82-90dad326ffad","Type":"ContainerStarted","Data":"aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d"} Oct 30 19:47:54.676049 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.675972 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f66d775b6-dq96j" event={"ID":"b569a9c0-5370-4ceb-9a82-90dad326ffad","Type":"ContainerStarted","Data":"5bbe69d136aefa4fd0c16be97fc6a022036f7086640c1a70cbcf76a8bc98f030"} Oct 30 19:47:54.828686 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:54.828606 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6f66d775b6-dq96j" podStartSLOduration=1.828588034 podStartE2EDuration="1.828588034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:54.826539788 +0000 UTC m=+64.986720997" watchObservedRunningTime="2026-10-30 19:47:54.828588034 +0000 UTC m=+64.988769232" Oct 30 19:47:55.066148 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:55.066120 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:55.374686 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:55.374586 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:55.723914 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:55.723866 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:56.021874 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.021764 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:56.131577 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.131203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:56.131577 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.131257 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:56.160413 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.160390 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 19:47:56.160523 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.160485 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 19:47:56.164013 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.163964 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:56.164013 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.164000 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:56.180956 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.180924 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:56.232226 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.232192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:56.247574 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.247549 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 19:47:56.267696 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.267671 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 19:47:56.275445 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.275392 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:56.424336 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.424301 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wnngx\"" Oct 30 19:47:56.429358 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.429337 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:47:56.434685 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.434662 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2gzs4\"" Oct 30 19:47:56.439626 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.439607 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:47:56.439869 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.439835 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mbspd\"" Oct 30 19:47:56.444793 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.444761 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:47:56.745134 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.745106 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jvsfx"] Oct 30 19:47:56.753072 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:56.753048 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3757b61_1b91_4739_a7fc_ecbfcb4feea6.slice/crio-725f1a5800106db465ded6b84f4c801a845e2c9681d0bc801cdab9aae29c51b1 WatchSource:0}: Error finding container 725f1a5800106db465ded6b84f4c801a845e2c9681d0bc801cdab9aae29c51b1: Status 404 returned error can't find the container with id 725f1a5800106db465ded6b84f4c801a845e2c9681d0bc801cdab9aae29c51b1 Oct 30 19:47:56.774856 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.774779 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bvdgb"] Oct 30 19:47:56.776557 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:56.776497 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62561598_09a8_4cb2_b04d_81ce7c0b8411.slice/crio-e1c56ad3d2439ca91bfaa5f7cec666ec8973391cf901b1f202f92fbbd3af7ab2 WatchSource:0}: Error finding container e1c56ad3d2439ca91bfaa5f7cec666ec8973391cf901b1f202f92fbbd3af7ab2: Status 404 returned error can't find the container with id e1c56ad3d2439ca91bfaa5f7cec666ec8973391cf901b1f202f92fbbd3af7ab2 Oct 30 19:47:56.776722 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:56.776696 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lb8sw"] Oct 30 19:47:56.779564 ip-10-0-142-168 kubenswrapper[2577]: W1030 19:47:56.779540 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda64e35d_dc35_4bc8_89cb_23a7eaf75836.slice/crio-63ee5421345e41a46e0021a3009c43bbd3e4643cbad13bd2386f977af175b314 WatchSource:0}: Error finding container 63ee5421345e41a46e0021a3009c43bbd3e4643cbad13bd2386f977af175b314: Status 404 returned error can't find the container with id 63ee5421345e41a46e0021a3009c43bbd3e4643cbad13bd2386f977af175b314 Oct 30 19:47:57.690877 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:57.690838 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bvdgb" event={"ID":"da64e35d-dc35-4bc8-89cb-23a7eaf75836","Type":"ContainerStarted","Data":"63ee5421345e41a46e0021a3009c43bbd3e4643cbad13bd2386f977af175b314"} Oct 30 19:47:57.694652 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:57.694582 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jvsfx" event={"ID":"a3757b61-1b91-4739-a7fc-ecbfcb4feea6","Type":"ContainerStarted","Data":"725f1a5800106db465ded6b84f4c801a845e2c9681d0bc801cdab9aae29c51b1"} Oct 30 19:47:57.698277 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:57.698249 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lb8sw" event={"ID":"62561598-09a8-4cb2-b04d-81ce7c0b8411","Type":"ContainerStarted","Data":"e1c56ad3d2439ca91bfaa5f7cec666ec8973391cf901b1f202f92fbbd3af7ab2"} Oct 30 19:47:58.077663 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:58.077633 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:58.327328 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:58.323941 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:58.705323 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:58.705279 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lb8sw" event={"ID":"62561598-09a8-4cb2-b04d-81ce7c0b8411","Type":"ContainerStarted","Data":"6eb14a3eada90f975bcfef9dca33e53c317914701a59241fdacafa01f13eac6b"} Oct 30 19:47:58.705491 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:58.705346 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lb8sw" event={"ID":"62561598-09a8-4cb2-b04d-81ce7c0b8411","Type":"ContainerStarted","Data":"d01f28258a2f8d6b338a7024c3bbfaad6f115ded96417c5c67af773b5172c1d7"} Oct 30 19:47:58.709663 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:58.709578 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerStarted","Data":"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430"} Oct 30 19:47:58.709793 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:58.709710 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerStarted","Data":"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49"} Oct 30 19:47:58.792772 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:47:58.792720 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lb8sw" podStartSLOduration=67.5049915 podStartE2EDuration="1m8.792704308s" totalImagesPullingTime="1.287712808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:58.775567649 +0000 UTC m=+68.935748848" watchObservedRunningTime="2026-10-30 19:47:58.792704308 +0000 UTC m=+68.952885504" Oct 30 19:48:00.162840 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:00.162732 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:00.413483 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:00.413396 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:01.760955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:01.760782 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerStarted","Data":"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449"} Oct 30 19:48:01.760955 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:01.760861 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerStarted","Data":"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795"} Oct 30 19:48:01.764976 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:01.764160 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bvdgb" event={"ID":"da64e35d-dc35-4bc8-89cb-23a7eaf75836","Type":"ContainerStarted","Data":"aa6edc294252d482ab6d11f54e33b4642473c71b1c32068c91156a4f6f277c56"} Oct 30 19:48:01.764976 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:01.764920 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:48:01.803881 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:01.803851 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:01.827206 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:01.826177 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bvdgb" podStartSLOduration=67.079747756 podStartE2EDuration="1m11.826157662s" totalImagesPullingTime="4.746409906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:01.824925242 +0000 UTC m=+71.985106441" watchObservedRunningTime="2026-10-30 19:48:01.826157662 +0000 UTC m=+71.986338859" Oct 30 19:48:02.154037 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:02.153999 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:02.777970 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:02.777933 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerStarted","Data":"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5"} Oct 30 19:48:02.778431 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:02.778053 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerStarted","Data":"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2"} Oct 30 19:48:02.892606 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:02.892538 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.141884726 podStartE2EDuration="10.892518995s" totalImagesPullingTime="6.735063502s" totalInitContainerRuntime="15.570767ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:02.878104429 +0000 UTC m=+73.038285628" watchObservedRunningTime="2026-10-30 19:48:02.892518995 +0000 UTC m=+73.052700192" Oct 30 19:48:03.074756 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:03.074674 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:03.300338 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:03.300241 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:03.351980 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:03.351949 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:03.537772 ip-10-0-142-168 kubenswrapper[2577]: I1030 19:48:03.537673 2577 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:48:03.537858 ip-10-0-142-168 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 19:48:03.548598 ip-10-0-142-168 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 19:48:03.548772 ip-10-0-142-168 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 19:48:03.548819 ip-10-0-142-168 systemd[1]: kubelet.service: Consumed 4.396s CPU time. Oct 30 19:48:03.566542 ip-10-0-142-168 systemd[1]: Starting Kubernetes Kubelet... Oct 30 19:48:03.631792 ip-10-0-142-168 kubenswrapper[6411]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:48:03.631792 ip-10-0-142-168 kubenswrapper[6411]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 19:48:03.631792 ip-10-0-142-168 kubenswrapper[6411]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:48:03.631792 ip-10-0-142-168 kubenswrapper[6411]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:48:03.633905 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633811 6411 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633913 6411 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633917 6411 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633922 6411 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633925 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633928 6411 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633931 6411 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633933 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633936 6411 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633938 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633941 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633944 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:48:03.633942 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633947 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633950 6411 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633953 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633956 6411 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633958 6411 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633961 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633963 6411 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633966 6411 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633972 6411 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633975 6411 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633978 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633981 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633983 6411 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633986 6411 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633988 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633991 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633993 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633996 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.633998 6411 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634000 6411 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:48:03.634235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634003 6411 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634006 6411 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634009 6411 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634012 6411 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634014 6411 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634017 6411 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634020 6411 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634022 6411 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634025 6411 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634027 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634030 6411 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634033 6411 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634036 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634038 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634041 6411 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634043 6411 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634046 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634049 6411 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634053 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634058 6411 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:48:03.634699 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634062 6411 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634067 6411 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634071 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634075 6411 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634079 6411 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634083 6411 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634089 6411 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634117 6411 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634120 6411 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634123 6411 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634126 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634129 6411 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634131 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634134 6411 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634137 6411 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634139 6411 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634142 6411 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634144 6411 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634147 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634150 6411 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:48:03.635197 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634153 6411 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634157 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634160 6411 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634163 6411 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634166 6411 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634169 6411 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634171 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634174 6411 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634176 6411 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634179 6411 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634182 6411 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634184 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634188 6411 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634192 6411 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634195 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634197 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634200 6411 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634202 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634205 6411 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:48:03.635673 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634208 6411 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634210 6411 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634213 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634215 6411 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634218 6411 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634220 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634223 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634226 6411 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634228 6411 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634231 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634234 6411 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634236 6411 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634239 6411 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634242 6411 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.634251 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635046 6411 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635055 6411 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635058 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635061 6411 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635064 6411 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:48:03.636143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635067 6411 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635069 6411 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635072 6411 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635075 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635078 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635080 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635083 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635085 6411 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635088 6411 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635091 6411 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635093 6411 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635097 6411 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635099 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635102 6411 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635104 6411 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635107 6411 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635109 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635111 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635114 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635116 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:48:03.636644 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635119 6411 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635121 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635124 6411 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635126 6411 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635129 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635131 6411 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635134 6411 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635137 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635139 6411 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635142 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635145 6411 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635147 6411 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635150 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635152 6411 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635155 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635157 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635160 6411 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635162 6411 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635165 6411 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635167 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:48:03.637143 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635171 6411 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635173 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635176 6411 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635178 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635181 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635183 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635186 6411 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635188 6411 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635191 6411 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635195 6411 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635198 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635201 6411 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635204 6411 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635207 6411 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635209 6411 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635212 6411 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635217 6411 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635221 6411 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635224 6411 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:48:03.637711 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635227 6411 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635231 6411 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635233 6411 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635236 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635238 6411 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635241 6411 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635245 6411 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635247 6411 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635250 6411 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635253 6411 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635255 6411 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635258 6411 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635260 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635263 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635265 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635268 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635270 6411 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635273 6411 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635275 6411 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:48:03.638235 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635278 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635280 6411 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635282 6411 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635285 6411 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635287 6411 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635289 6411 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635292 6411 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635294 6411 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635297 6411 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635299 6411 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635302 6411 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635304 6411 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635307 6411 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635310 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635312 6411 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635315 6411 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635318 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635320 6411 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635323 6411 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:48:03.638694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635325 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635328 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635330 6411 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635333 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635918 6411 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635927 6411 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635930 6411 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635934 6411 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635937 6411 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635940 6411 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635943 6411 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635945 6411 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635948 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635950 6411 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635953 6411 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635956 6411 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635958 6411 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635961 6411 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635963 6411 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635966 6411 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:48:03.639350 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635968 6411 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635971 6411 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635973 6411 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635975 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635978 6411 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635981 6411 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635984 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635986 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635995 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.635998 6411 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636001 6411 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636003 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636006 6411 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636008 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636011 6411 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636013 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636016 6411 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636018 6411 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636021 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:48:03.639832 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636023 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636026 6411 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636029 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636031 6411 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636033 6411 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636036 6411 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636038 6411 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636041 6411 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636043 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636046 6411 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636048 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636050 6411 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636053 6411 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636055 6411 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636057 6411 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636059 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636063 6411 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636067 6411 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636070 6411 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:48:03.640343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636073 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636075 6411 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636078 6411 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636085 6411 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636088 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636091 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636093 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636095 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636098 6411 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636100 6411 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636102 6411 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636105 6411 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636107 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636112 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636114 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636117 6411 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636119 6411 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636123 6411 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636126 6411 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:48:03.640806 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636130 6411 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636133 6411 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636136 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636138 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636142 6411 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636144 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636147 6411 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636149 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636152 6411 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636155 6411 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636157 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636160 6411 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636163 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636165 6411 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636168 6411 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636171 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636173 6411 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636183 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636186 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636188 6411 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:48:03.641288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636191 6411 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636194 6411 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636196 6411 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636198 6411 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636201 6411 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636203 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636209 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636211 6411 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636214 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636217 6411 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636219 6411 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636223 6411 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.636225 6411 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636343 6411 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636355 6411 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636364 6411 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636369 6411 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636374 6411 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636377 6411 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636382 6411 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636386 6411 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 19:48:03.641765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636389 6411 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636392 6411 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636395 6411 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636398 6411 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636401 6411 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636404 6411 flags.go:64] FLAG: --cgroup-root="" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636407 6411 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636409 6411 flags.go:64] FLAG: --client-ca-file="" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636412 6411 flags.go:64] FLAG: --cloud-provider="external" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636424 6411 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636430 6411 flags.go:64] FLAG: --cluster-domain="" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636433 6411 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636436 6411 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636439 6411 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636443 6411 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636447 6411 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636450 6411 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636459 6411 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636462 6411 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636464 6411 flags.go:64] FLAG: --contention-profiling="false" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636468 6411 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636470 6411 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636474 6411 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636478 6411 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 19:48:03.642591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636482 6411 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636485 6411 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636488 6411 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636491 6411 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636494 6411 flags.go:64] FLAG: --enable-server="true" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636497 6411 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636504 6411 flags.go:64] FLAG: --event-burst="100" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636507 6411 flags.go:64] FLAG: --event-qps="50" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636510 6411 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636513 6411 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636516 6411 flags.go:64] FLAG: --eviction-hard="" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636520 6411 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636523 6411 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636526 6411 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636529 6411 flags.go:64] FLAG: --eviction-soft="" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636532 6411 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636535 6411 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636538 6411 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636546 6411 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636549 6411 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636552 6411 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636555 6411 flags.go:64] FLAG: --feature-gates="" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636559 6411 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636562 6411 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636565 6411 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 19:48:03.643446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636570 6411 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636573 6411 flags.go:64] FLAG: --healthz-port="10248" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636576 6411 flags.go:64] FLAG: --help="false" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636579 6411 flags.go:64] FLAG: --hostname-override="ip-10-0-142-168.ec2.internal" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636582 6411 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636585 6411 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636592 6411 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636595 6411 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636598 6411 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636601 6411 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636604 6411 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636607 6411 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636610 6411 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636612 6411 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636615 6411 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636618 6411 flags.go:64] FLAG: --kube-reserved="" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636621 6411 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636624 6411 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636627 6411 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636629 6411 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636632 6411 flags.go:64] FLAG: --lock-file="" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636635 6411 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636638 6411 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636641 6411 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 19:48:03.644154 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636647 6411 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636650 6411 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636659 6411 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636662 6411 flags.go:64] FLAG: --logging-format="text" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636664 6411 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636668 6411 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636670 6411 flags.go:64] FLAG: --manifest-url="" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636673 6411 flags.go:64] FLAG: --manifest-url-header="" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636678 6411 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636681 6411 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636685 6411 flags.go:64] FLAG: --max-pods="110" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636687 6411 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636690 6411 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636693 6411 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636697 6411 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636700 6411 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636703 6411 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636706 6411 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636715 6411 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636718 6411 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636721 6411 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636724 6411 flags.go:64] FLAG: --pod-cidr="" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636727 6411 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636729 6411 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636732 6411 flags.go:64] FLAG: --pods-per-core="0" Oct 30 19:48:03.644762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636735 6411 flags.go:64] FLAG: --port="10250" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636738 6411 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636741 6411 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b2c1ccb4c90b384" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636744 6411 flags.go:64] FLAG: --qos-reserved="" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636747 6411 flags.go:64] FLAG: --read-only-port="10255" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636750 6411 flags.go:64] FLAG: --register-node="true" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636753 6411 flags.go:64] FLAG: --register-with-taints="" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636756 6411 flags.go:64] FLAG: --registry-burst="10" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636759 6411 flags.go:64] FLAG: --registry-qps="5" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636762 6411 flags.go:64] FLAG: --reserved-cpus="" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636765 6411 flags.go:64] FLAG: --reserved-memory="" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636768 6411 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636771 6411 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636775 6411 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636778 6411 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636780 6411 flags.go:64] FLAG: --runonce="false" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636783 6411 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636787 6411 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636789 6411 flags.go:64] FLAG: --seccomp-default="false" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636792 6411 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636795 6411 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636800 6411 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636803 6411 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636806 6411 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636809 6411 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636811 6411 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 19:48:03.645507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636814 6411 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636817 6411 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636820 6411 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636823 6411 flags.go:64] FLAG: --system-cgroups="" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636825 6411 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636831 6411 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636834 6411 flags.go:64] FLAG: --tls-cert-file="" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636837 6411 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636843 6411 flags.go:64] FLAG: --tls-min-version="" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636846 6411 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636849 6411 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636852 6411 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636854 6411 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636857 6411 flags.go:64] FLAG: --v="2" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636861 6411 flags.go:64] FLAG: --version="false" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636865 6411 flags.go:64] FLAG: --vmodule="" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636869 6411 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.636873 6411 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637042 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637046 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637049 6411 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637052 6411 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637054 6411 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637057 6411 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:48:03.646278 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637060 6411 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637062 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637065 6411 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637069 6411 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637072 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637075 6411 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637077 6411 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637080 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637082 6411 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637085 6411 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637087 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637090 6411 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637092 6411 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637095 6411 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637097 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637100 6411 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637102 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637105 6411 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637107 6411 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637110 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:48:03.646856 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637112 6411 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637115 6411 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637117 6411 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637119 6411 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637122 6411 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637124 6411 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637127 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637130 6411 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637132 6411 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637135 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637137 6411 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637140 6411 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637143 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637146 6411 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637148 6411 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637153 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637155 6411 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637158 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637162 6411 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:48:03.647349 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637164 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637166 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637169 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637171 6411 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637174 6411 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637178 6411 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637181 6411 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637184 6411 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637187 6411 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637190 6411 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637193 6411 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637195 6411 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637197 6411 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637200 6411 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637203 6411 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637205 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637207 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637210 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637213 6411 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:48:03.647799 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637215 6411 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637218 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637220 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637223 6411 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637225 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637228 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637230 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637232 6411 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637235 6411 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637238 6411 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637241 6411 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637243 6411 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637247 6411 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637250 6411 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637252 6411 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637254 6411 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637257 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637260 6411 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637262 6411 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637265 6411 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:48:03.648288 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637267 6411 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637270 6411 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637272 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637275 6411 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637278 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637280 6411 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637283 6411 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637285 6411 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637287 6411 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637291 6411 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637295 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637299 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637302 6411 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637304 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637307 6411 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637310 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637312 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637315 6411 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637317 6411 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:48:03.648869 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637319 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637322 6411 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637325 6411 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.637351 6411 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637391 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637397 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637400 6411 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637403 6411 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637406 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637408 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637411 6411 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637413 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637416 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637419 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637421 6411 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:48:03.649543 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637423 6411 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637426 6411 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637429 6411 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637431 6411 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637434 6411 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637436 6411 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637438 6411 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637441 6411 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637443 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637446 6411 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637448 6411 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637451 6411 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637453 6411 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637457 6411 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637459 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637462 6411 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637464 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637467 6411 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637469 6411 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637471 6411 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:48:03.650069 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637475 6411 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637479 6411 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637484 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637488 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637491 6411 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637493 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637496 6411 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637498 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637500 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637503 6411 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637505 6411 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637508 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637510 6411 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637512 6411 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637515 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637517 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637520 6411 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637522 6411 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637524 6411 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:48:03.650826 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637527 6411 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637529 6411 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637532 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637535 6411 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637539 6411 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637542 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637547 6411 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637551 6411 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637555 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637559 6411 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637563 6411 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637567 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637571 6411 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637577 6411 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637581 6411 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637586 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637590 6411 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637595 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637599 6411 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637603 6411 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:48:03.651594 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637607 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637609 6411 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637612 6411 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637614 6411 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637617 6411 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637619 6411 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637622 6411 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637624 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637627 6411 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637629 6411 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637632 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637635 6411 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637637 6411 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637639 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637643 6411 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637646 6411 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637650 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637655 6411 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637659 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:48:03.652402 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637663 6411 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637668 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637671 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637673 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637676 6411 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637678 6411 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637684 6411 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637688 6411 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637693 6411 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637696 6411 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637699 6411 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637702 6411 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637705 6411 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637707 6411 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637711 6411 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637714 6411 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.637717 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.641696 6411 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.641713 6411 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 19:48:03.653171 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.641730 6411 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.641735 6411 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642589 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642608 6411 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642615 6411 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642626 6411 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642632 6411 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642637 6411 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642641 6411 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642645 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642650 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642653 6411 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642658 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642662 6411 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642667 6411 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642671 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642675 6411 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642695 6411 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642700 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:48:03.653940 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642703 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642708 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642712 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642716 6411 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642721 6411 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642725 6411 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642728 6411 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642732 6411 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642738 6411 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642742 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642746 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642755 6411 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642759 6411 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642764 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642768 6411 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642772 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642797 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642802 6411 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642806 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:48:03.654664 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642810 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642814 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642819 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642823 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642827 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642836 6411 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642840 6411 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642844 6411 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642848 6411 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642852 6411 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642856 6411 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642860 6411 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642864 6411 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642868 6411 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642873 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642876 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642881 6411 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642885 6411 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642917 6411 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642921 6411 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:48:03.655343 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642925 6411 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642930 6411 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642935 6411 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642938 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642942 6411 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.642947 6411 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643120 6411 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643466 6411 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643479 6411 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643485 6411 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643488 6411 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643492 6411 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643496 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643498 6411 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643501 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643503 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643507 6411 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643510 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643512 6411 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643515 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:48:03.655820 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643518 6411 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643521 6411 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643523 6411 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643526 6411 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643529 6411 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643531 6411 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643534 6411 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643536 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643539 6411 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643542 6411 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643545 6411 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643547 6411 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643550 6411 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643552 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643555 6411 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643558 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643560 6411 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643565 6411 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643570 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643573 6411 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:48:03.656373 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643576 6411 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643579 6411 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643581 6411 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643585 6411 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643588 6411 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643591 6411 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643594 6411 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643597 6411 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643602 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643605 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.643640 6411 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643832 6411 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643837 6411 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643840 6411 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643843 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:48:03.656967 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643846 6411 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643848 6411 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643851 6411 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643854 6411 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643856 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643860 6411 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643862 6411 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643865 6411 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643867 6411 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643870 6411 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643873 6411 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643875 6411 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643878 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643880 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643883 6411 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643886 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643888 6411 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643891 6411 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643918 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643924 6411 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:48:03.657339 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643928 6411 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643934 6411 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643940 6411 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643944 6411 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643948 6411 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643953 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643958 6411 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643962 6411 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643967 6411 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643971 6411 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643975 6411 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643978 6411 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643983 6411 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643986 6411 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643990 6411 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.643994 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644016 6411 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644022 6411 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644027 6411 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:48:03.657862 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644031 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644036 6411 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644040 6411 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644044 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644048 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644052 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644056 6411 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644060 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644065 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644069 6411 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644072 6411 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644077 6411 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644081 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644086 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644089 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644093 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644097 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644102 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644105 6411 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:48:03.658499 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644109 6411 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644113 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644117 6411 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644121 6411 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644125 6411 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644141 6411 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644144 6411 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644148 6411 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644152 6411 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644157 6411 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644161 6411 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644165 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644169 6411 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644182 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644187 6411 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644191 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644195 6411 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644199 6411 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644203 6411 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644208 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:48:03.659225 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644212 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644218 6411 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644222 6411 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644228 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644233 6411 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644237 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644242 6411 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644245 6411 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644249 6411 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644253 6411 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644257 6411 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644261 6411 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644264 6411 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644268 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644272 6411 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644275 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644279 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644283 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644286 6411 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644290 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:48:03.660040 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644293 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644297 6411 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644301 6411 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644305 6411 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.644358 6411 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644407 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644425 6411 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644429 6411 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644434 6411 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644438 6411 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644443 6411 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644447 6411 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644450 6411 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644455 6411 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644459 6411 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644463 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:48:03.660721 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644466 6411 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644470 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644474 6411 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644478 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644483 6411 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644488 6411 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644492 6411 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644496 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644500 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644504 6411 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644508 6411 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644512 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644516 6411 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644519 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644524 6411 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644528 6411 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644532 6411 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644536 6411 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644540 6411 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:48:03.661132 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644544 6411 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644548 6411 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644552 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644556 6411 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644567 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644572 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644576 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644580 6411 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644584 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644588 6411 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644592 6411 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644596 6411 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644601 6411 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644604 6411 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644609 6411 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644613 6411 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644616 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644620 6411 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644624 6411 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:48:03.661619 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644629 6411 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644633 6411 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644637 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644642 6411 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644646 6411 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644650 6411 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644654 6411 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644658 6411 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644663 6411 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644667 6411 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644671 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644675 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644678 6411 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644682 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644686 6411 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644690 6411 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644694 6411 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644698 6411 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644710 6411 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644716 6411 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:48:03.662097 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644722 6411 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644727 6411 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644735 6411 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644739 6411 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644746 6411 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644752 6411 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644757 6411 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644760 6411 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644765 6411 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644769 6411 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644773 6411 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644778 6411 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644782 6411 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644786 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644790 6411 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644795 6411 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644799 6411 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644803 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644807 6411 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644811 6411 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:48:03.662689 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644816 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644820 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644824 6411 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644830 6411 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644835 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644839 6411 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644843 6411 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644852 6411 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644856 6411 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644860 6411 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644865 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644878 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644883 6411 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644888 6411 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644913 6411 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644917 6411 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:03.644922 6411 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.645043 6411 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.646165 6411 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 19:48:03.663508 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.646224 6411 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.646433 6411 server.go:1014] "Starting client certificate rotation" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.646569 6411 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.646627 6411 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:41:50 +0000 UTC" deadline="2028-07-22 20:49:06.800270856 +0000 UTC" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.646662 6411 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15145h1m3.153612767s" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.646984 6411 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.648281 6411 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.649461 6411 log.go:25] "Validated CRI v1 runtime API" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.653814 6411 log.go:25] "Validated CRI v1 image API" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.654656 6411 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 19:48:03.664118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.656948 6411 fs.go:135] Filesystem UUIDs: map[2c6ad25c-b20b-46c6-9d11-c6c616e59775:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cccf4815-5b4f-4592-a081-750a3f31e5b9:/dev/nvme0n1p4] Oct 30 19:48:03.664420 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.656969 6411 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/108e4ff9a325e8336bd5555eff508483b2a9634189a3084567c4e224fd0bb00d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/108e4ff9a325e8336bd5555eff508483b2a9634189a3084567c4e224fd0bb00d/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1742198c1994027a8d5414071eef86a8e5ca5784dcdc1cad0a3052620a232b23/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1742198c1994027a8d5414071eef86a8e5ca5784dcdc1cad0a3052620a232b23/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/272abb5096c53f8c612c69d6d4490663181c3e3ac2359d3f89490ec57116fb67/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/272abb5096c53f8c612c69d6d4490663181c3e3ac2359d3f89490ec57116fb67/userdata/shm major:0 minor:316 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2d70e83de570fe12631bdfc08c7f42ccef4b37586478f4045968742581a1bc44/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2d70e83de570fe12631bdfc08c7f42ccef4b37586478f4045968742581a1bc44/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/38efa6c6146cff0df453fce2e0de42b581de3b4cb80b73e6c2279da08ddeec2d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/38efa6c6146cff0df453fce2e0de42b581de3b4cb80b73e6c2279da08ddeec2d/userdata/shm major:0 minor:260 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/410c008ae2e899f7593d50532d68df1c269ac4d3982eb1c761d236dbf79cee6d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/410c008ae2e899f7593d50532d68df1c269ac4d3982eb1c761d236dbf79cee6d/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4348d980cbcc3ed0b92fd34a2facffc6d4a6318e67cc2425df24d79218a2dc0a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4348d980cbcc3ed0b92fd34a2facffc6d4a6318e67cc2425df24d79218a2dc0a/userdata/shm major:0 minor:204 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4b96e6b037426a85d3a891398d5ccfa339192e04ce18ad4402cb8f4a9b6c81bd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4b96e6b037426a85d3a891398d5ccfa339192e04ce18ad4402cb8f4a9b6c81bd/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5bbe69d136aefa4fd0c16be97fc6a022036f7086640c1a70cbcf76a8bc98f030/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5bbe69d136aefa4fd0c16be97fc6a022036f7086640c1a70cbcf76a8bc98f030/userdata/shm major:0 minor:376 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/63ee5421345e41a46e0021a3009c43bbd3e4643cbad13bd2386f977af175b314/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/63ee5421345e41a46e0021a3009c43bbd3e4643cbad13bd2386f977af175b314/userdata/shm major:0 minor:393 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/66c35a3180c32bebd160df0dfb67293ffaac1bb6aea0e22cd499740e35bad0d4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/66c35a3180c32bebd160df0dfb67293ffaac1bb6aea0e22cd499740e35bad0d4/userdata/shm major:0 minor:89 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/67aca29c1952f4205820a503a15429729f5c93cee8bfbb5ca5e638d824be5ebc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/67aca29c1952f4205820a503a15429729f5c93cee8bfbb5ca5e638d824be5ebc/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/725f1a5800106db465ded6b84f4c801a845e2c9681d0bc801cdab9aae29c51b1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/725f1a5800106db465ded6b84f4c801a845e2c9681d0bc801cdab9aae29c51b1/userdata/shm major:0 minor:389 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8498a05bd0dea1a76f393f6262913bac53039ce037a8ac8c03682da8af01858a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8498a05bd0dea1a76f393f6262913bac53039ce037a8ac8c03682da8af01858a/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/84cf4865b87984598fe3b16ba807871a01d29dd831fb876a67a26dd750d45fa6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/84cf4865b87984598fe3b16ba807871a01d29dd831fb876a67a26dd750d45fa6/userdata/shm major:0 minor:253 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/979ecd3c5b31a7cde5cc776ac33c2cfa5c47782725662283a7f08baa058fc829/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/979ecd3c5b31a7cde5cc776ac33c2cfa5c47782725662283a7f08baa058fc829/userdata/shm major:0 minor:85 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b6d4cafc1eb47eab5e59ca7ae4f78a7d63dd9850f52f470fa55a6d9aa65185ce/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b6d4cafc1eb47eab5e59ca7ae4f78a7d63dd9850f52f470fa55a6d9aa65185ce/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b932d03caaef8b6a30aba82ea26989b0a0e4f4c7849e38bfe5671eb540886c72/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b932d03caaef8b6a30aba82ea26989b0a0e4f4c7849e38bfe5671eb540886c72/userdata/shm major:0 minor:369 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c3ae80f992f83041d46b23c1ad5bd0cc8a3ce485ad9d199b83d0e12f8da4ccc4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c3ae80f992f83041d46b23c1ad5bd0cc8a3ce485ad9d199b83d0e12f8da4ccc4/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d03e243eafcb3ce5236c92e0eb4fe70065a94c06f91e1cba3ea5b8be5a092566/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d03e243eafcb3ce5236c92e0eb4fe70065a94c06f91e1cba3ea5b8be5a092566/userdata/shm major:0 minor:305 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d70ff94cd072c48e42ad24591d42f075373f1326410d83b4e8b19d99a659c1c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d70ff94cd072c48e42ad24591d42f075373f1326410d83b4e8b19d99a659c1c1/userdata/shm major:0 minor:100 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e1c56ad3d2439ca91bfaa5f7cec666ec8973391cf901b1f202f92fbbd3af7ab2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e1c56ad3d2439ca91bfaa5f7cec666ec8973391cf901b1f202f92fbbd3af7ab2/userdata/shm major:0 minor:391 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f2f79ed46b081f577502da86affe4b5f13e7c71a6535249600fba0757a9204e2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f2f79ed46b081f577502da86affe4b5f13e7c71a6535249600fba0757a9204e2/userdata/shm major:0 minor:284 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/018f82d1-73aa-4c50-bce5-6fef86b947b0/volumes/kubernetes.io~projected/kube-api-access-sm7d6:{mountpoint:/var/lib/kubelet/pods/018f82d1-73aa-4c50-bce5-6fef86b947b0/volumes/kubernetes.io~projected/kube-api-access-sm7d6 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~projected/kube-api-access-qdxr6:{mountpoint:/var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~projected/kube-api-access-qdxr6 major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/27586708-df16-4c22-9b2c-eb87fc94595d/volumes/kubernetes.io~projected/kube-api-access-sdvhz:{mountpoint:/var/lib/kubelet/pods/27586708-df16-4c22-9b2c-eb87fc94595d/volumes/kubernetes.io~projected/kube-api-access-sdvhz major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/27586708-df16-4c22-9b2c-eb87fc94595d/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/27586708-df16-4c22-9b2c-eb87fc94595d/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/298e0d65-779b-4fd0-8931-43596a15d5fa/volumes/kubernetes.io~projected/kube-api-access-grrnx:{mountpoint:/var/lib/kubelet/pods/298e0d65-779b-4fd0-8931-43596a15d5fa/volumes/kubernetes.io~projected/kube-api-access-grrnx major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3009ff95-7457-4856-be99-e3891d923002/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/3009ff95-7457-4856-be99-e3891d923002/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3864b62c-1c37-4c73-89c5-136a1f402078/volumes/kubernetes.io~projected/kube-api-access-hm4wj:{mountpoint:/var/lib/kubelet/pods/3864b62c-1c37-4c73-89c5-136a1f402078/volumes/kubernetes.io~projected/kube-api-access-hm4wj major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~projected/kube-api-access-4z65z:{mountpoint:/var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~projected/kube-api-access-4z65z major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46a18c80-179d-46ba-85aa-1489814be80e/volumes/kubernetes.io~projected/kube-api-access-2g5bf:{mountpoint:/var/lib/kubelet/pods/46a18c80-179d-46ba-85aa-1489814be80e/volumes/kubernetes.io~projected/kube-api-access-2g5bf major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46a18c80-179d-46ba-85aa-1489814be80e/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/46a18c80-179d-46ba-85aa-1489814be80e/volumes/kubernetes.io~secret/metrics-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/alertmanager/9 major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/config-reloader/6 major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~empty-dir/config-out major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~projected/kube-api-access-pmc7x:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~projected/kube-api-access-pmc7x major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~projected/tls-assets major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/config-volume major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/web-config major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62561598-09a8-4cb2-b04d-81ce7c0b8411/volumes/kubernetes.io~projected/kube-api-access-nbgzs:{mountpoint:/var/lib/kubelet/pods/62561598-09a8-4cb2-b04d-81ce7c0b8411/volumes/kubernetes.io~projected/kube-api-access-nbgzs major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62561598-09a8-4cb2-b04d-81ce7c0b8411/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/62561598-09a8-4cb2-b04d-81ce7c0b8411/volumes/kubernetes.io~secret/metrics-certs major:0 minor:382 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65d4fb96-8b10-487d-9912-bd71dd507120/volumes/kubernetes.io~projected/kube-api-access-c7qdn:{mountpoint:/var/lib/kubelet/pods/65d4fb96-8b10-487d-9912-bd71dd507120/volumes/kubernetes.io~projected/kube-api-access-c7qdn major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65d4fb96-8b10-487d-9912-bd71dd507120/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/65d4fb96-8b10-487d-9912-bd71dd507120/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/config-reloader/2 major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/prometheus/15 major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~empty-dir/config-out major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~projected/kube-api-access-wzrnv:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~projected/kube-api-access-wzrnv major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~projected/tls-assets major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/config major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:365 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:359 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:353 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:358 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:366 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/web-config major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e/volumes/kubernetes.io~projected/kube-api-access-s49fn:{mountpoint:/var/lib/kubelet/pods/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e/volumes/kubernetes.io~projected/kube-api-access-s49fn major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3757b61-1b91-4739-a7fc-ecbfcb4feea6/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/a3757b61-1b91-4739-a7fc-ecbfcb4feea6/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:387 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~projected/kube-api-access-n8lff:{mountpoint:/var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~projected/kube-api-access-n8lff major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~projected/kube-api-access-glwp6:{mountpoint:/var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~projected/kube-api-access-glwp6 major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:373 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb90a295-9822-4874-9014-0a91ac7a73c5/volumes/kubernetes.io~projected/kube-api-access-kpw8x:{mountpoint:/var/lib/kubelet/pods/bb90a295-9822-4874-9014-0a91ac7a73c5/volumes/kubernetes.io~projected/kube-api-access-kpw8x major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57/volumes/kubernetes.io~projected/kube-api-access-f4w9j:{mountpoint:/var/lib/kubelet/pods/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57/volumes/kubernetes.io~projected/kube-api-access-f4w9j major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~projected/kube-api-access-t4w2g:{mountpoint:/var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~projected/kube-api-access-t4w2g major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:246 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/da64e35d-dc35-4bc8-89cb-23a7eaf75836/volumes/kubernetes.io~projected/kube-api-access-wdmfq:{mountpoint:/var/lib/kubelet/pods/da64e35d-dc35-4bc8-89cb-23a7eaf75836/volumes/kubernetes.io~projected/kube-api-access-wdmfq major:0 minor:388 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~projected/kube-api-access-vkzlx:{mountpoint:/var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~projected/kube-api-access-vkzlx major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volumes/kubernetes.io~projected/kube-api-access-fzn4q:{mountpoint:/var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volumes/kubernetes.io~projected/kube-api-access-fzn4q major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/eb07973c43264fd4302ec65953e16a184d2c238f81d8dfd604f85e9e8ba38f54/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/d7a996b429d93cf845355a6f6acc4ced97291d192e11bc525e81f3f3c8984ae5/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/fed03b646c294c871d18b2eed231706c54f74d6ea094b822070f6ed9a6020922/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/5fb675983b1ddddd66289a72e0348a4fa9f4fc995dd711f02af799e9e38943e8/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/f62a623a62140b6202b919e650c11f11bd7e896f00617d436fac89c6ce95de14/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/6b3f29d79251e1191b9fa0a6d0c1e6ad55c1ff4a0631d6993eef33e72a14415f/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/adcaef8757b8cc0901451eb24e3dcfdfa3534b44e213dbd1b035c876aab7bdad/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/b82988586d09606a006f508039987985fdee6aaf09b5ebba5031987edac011cf/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/01e9b18705a4a38f7a374038add9a0980115b359aac793a076c27a2edb78196a/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/b126868c32ae5749a23f8a6484f5963c81e83de7ad75dcf8ed44a46fca3756ca/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/7d371ab1f2c9d992b5c79df174332b412e3a937cd3b941243f5598e6993527fa/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/90f27a820ba5b4b39fdd0141806f5fd7f77a123f1376f27bce1082b9df2f3edc/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/5b46fa687239c03db0eb0286a7b03da63c3b9902da780095fb25aa7406197efb/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/0ccda2ab9688d15b7c6992bf366d29621ddf262ce78fef4e1e09ba3a8e40e91a/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/7f998181901ad3c8e362541382f52fe9351adae238d11d160cdb4254e55c912a/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/347ec0c379d0164c1d6b0ddd1c0a6d0e244217c9eb0c745ccaad255de180a691/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d50775d1246299fe8cf68d8bac0d145e3e7aa2eabd3cd48b5b55e368426ae5bc/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-179:{mountpoint:/var/lib/containers/storage/overlay/0f288c40b93c3c57eafd9fd2e977f5a0f8a2f1760e367454a9372d94406b48e4/merged major:0 minor:179 fsType:overlay blockSize:0} overlay_0-184:{mountpoint:/var/lib/containers/storage/overlay/248f7ba8f356a75ede53a01fbf1e1a4fe67cf02b930a68952654bfc07ded7fa2/merged major:0 minor:184 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/b3b11f2e06ba4f7ab5de364a96a363d4b742ebc8e1b75d8cd0d85f0c1f90b0c6/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/e8550144952b4bffd72513e8f9490d8b56467f9c1589e152374e354a88263db8/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/b654acd88eae76433f930b2d0c7d4cdfec0ba7ffb43071c0384cf2da18bb2e09/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/aa598de84e98bba50233de65ac5f1c00ead3946e99b3b2c4d73c70f498853516/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/7416fb8a38d6ce85979c20ed7bb5d91286eaaa72f1719f2f532cf6aeb31879ac/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/04677d43eb70979bfb07fa879069976a4d5d1c3132fd5629a11c561e584d5bf2/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/64dd5e2e061ef6e20ea5f26bed2908dc9693973b4562b1e03a56ecc8c9a5910d/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/37a0bdacf31076569de6b4bfb56ed622f0fc019f3760fe5d56c0d7582584d23a/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/ca01ac3025bff37bee428e8e3d742a87a3279a7a4a3e4b5174f97b955421911c/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/9ca0d829fbc92d45c02d85ae771d9e9ecc51e52cdfb1a150530840e41778c446/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/a6c0f7fdb360525c6349f4457dfbf3e5aaeec19ca59b51d750a871482fa605ce/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/6ee72631c4ea09c7564371050750fa701213fe47f34e00be0cc127b2f760b2f1/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/3f8a946f834900cc32a8e99913cca4ca20b28455e3fadd74940cf0ed3d2d24e2/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/be561c28bd70d7acd48b0475b41702a3102246e953a03b85c63a873a85d574ff/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/400221243a22000f81bfac09e5b3d7c183dfa702bbec339f847ccf8d70e2068a/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/740cfffad064b53b105d7cd6de24d41055770025896556ff57793f96fd4ad2f5/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/e12793c2e5d032dadf51cf8a754ec22aef2505709305d742a67479eb14dd3efe/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/08abdaf70b5fa527da5c10c400e68dc8c7748ae388fb08c9841e2944490134ed/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/f5fcc07152389b1ad48b97acfe4184708611e7d5261367e3f0fbee1fa2aec2f3/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/dceff2ff79e334984dd5457a01ac2c84fad1dd1dbb20a6da0d6f1fb126f8267a/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/b0fe9bfd6f458d81408814382ee2a04a4b61c697390ec9d417841aa982e3883a/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/11ee9f262249a34757845cf73d02a1680fce27e4c7ee9fecde18232bc8efdfde/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/69a84cf2a35e7ce59043b3338b4ebdf0d0989f86648acb438cdb2b7ce49aef1d/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-333:{mountpoint:/var/lib/containers/storage/overlay/d524266e449f8abf4e219666c6c9b01c3010577e7740d9d57459efcfafc3f4ff/merged major:0 minor:333 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/0b55af02b8fffd564119328b8684b063f92e0fca20a4d88cad078b61a94dc221/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/6919f45dd08443160934976a5e915e56e6556aed690c24dd7c1995139ada4dd9/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/f77f9e602ad5400b2833ea7a03662502e0ee030f243a9417614d703955a49b4c/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-371:{mountpoint:/var/lib/containers/storage/overlay/ce4f737d18dda92b73ca380b80c8803b22b6cf0d5dc09468e85fd4baf8627115/merged major:0 minor:371 fsType:overlay blockSize:0} overlay_0-378:{mountpoint:/var/lib/containers/storage/overlay/4d25b42a07ffa495ab86a9b6abc6f8c39778e230c4f5cbfa3b6df200f6b919d8/merged major:0 minor:378 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/bcb4e699745a858b64e5f5fa5985239c3100da27b21f0ccd87c7896d3e088fcb/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-395:{mountpoint:/var/lib/containers/storage/overlay/91227ca61b2e48f65792e52f309b6b8c636fcadf35715a8e46603603bb47c52c/merged major:0 minor:395 fsType:overlay blockSize:0} overlay_0-397:{mountpoint:/var/lib/containers/storage/overlay/093d7f93bca55a5437ac8111b62275d99f01cf881ab7411093053762aa3a3918/merged major:0 minor:397 fsType:overlay blockSize:0} overlay_0-399:{mountpoint:/var/lib/containers/storage/overlay/ed7b24c67dcb46514fb0fd055994a618f3c7c2b685fffd2fac8eb07e9417e4aa/merged major:0 minor:399 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/e6f3b52a2ff5bdb4c56d1d15183bea977fe89815361cac78d0b9a88cb3b3779d/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-403:{mountpoint:/var/lib/containers/storage/overlay/7c06436789046956576a05bf182b7374318b964ba60b0f1d2a1425bb2b2a86b5/merged major:0 minor:403 fsType:overlay blockSize:0} overlay_0-409:{mountpoint:/var/lib/containers/storage/overlay/da2c0c69422aef6e1d1736b2f90d431dd639bb6620e5d9496fdbb11be7eebd83/merged major:0 minor:409 fsType:overlay blockSize:0} overlay_0-414:{mountpoint:/var/lib/containers/storage/overlay/af877fc57782042622fead65954033c960fd6206ab088c22e2d569c7bd481d6e/merged major:0 minor:414 fsType:overlay blockSize:0} overlay_0-420:{mountpoint:/var/lib/containers/storage/overlay/06b2a3ad199837697b95a783e206c62a0062bd3374632e508129f9ac55bd5759/merged major:0 minor:420 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/6af39eae28abba7dc1fcced4255100c47d3410630c583b64e981c7eeeaedf6fe/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-427:{mountpoint:/var/lib/containers/storage/overlay/7684b43ed90e3a8394ec82edbd4e0a0080c6b0019cff58b86ab105fb92c92545/merged major:0 minor:427 fsType:overlay blockSize:0} overlay_0-436:{mountpoint:/var/lib/containers/storage/overlay/903f12720355352c77bb746ef8accc504fce74b828802e7e7850f102e747fe6b/merged major:0 minor:436 fsType:overlay blockSize:0} overlay_0-441:{mountpoint:/var/lib/containers/storage/overlay/6729f39caf9217595b53cd626d0bcb7e4edabd4b532268e6f9c2252f5c11d37f/merged major:0 minor:441 fsType:overlay blockSize:0} overlay_0-446:{mountpoint:/var/lib/containers/storage/overlay/a67575171a7d05f847c673ab949fa1ff31208ee0e12585f61968a05743ea5255/merged major:0 minor:446 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/0c1e1597329ed395767bf4687bbfc3174a8dd63a2734187f4611418ee7b08888/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/09f6c7cc3c9f9c91af51208ff8e3fbfd2d24fa1eeecca3f5f376d27a2132bcd5/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/f9b1fb7c73d53a56e61b52d7770d493f8ad272c9209315ef3d3bb6de52ac6171/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/e3b62c18fdeb45723b7cca8e637e56521f3b91fc43a03d9d663c56697de12ca7/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/b2f2b92b48cf5b16dcb1b700e2dfb6aa48e18d5a353e0ac364e75b563ce1ed59/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/ee64cf8ae8aae3712892e1a5007548433d2c12e9ec56e2b3ae8ef68ddb3cd839/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/4dea373076817acdb6a176b61a3d6e7adacb882d34724ee9c6c6f883b0ed4f2e/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/69ecb7839d23101bc439d97f476918f6f7b6393fb562e45bab1b9abdf9e4d038/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-91:{mountpoint:/var/lib/containers/storage/overlay/f330d1946fe501935525bb86573e39563a6e0ef97e0e1538c4bf48938d3cbe12/merged major:0 minor:91 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/41fdb021db5332b87d39d08648c3417e59cb8b5e4ff3b956f3a47eb3154cdd2d/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/3d1a71821b63bb9b0218586188a64300b9c15375dcd46a30b654d0dcba58bc54/merged major:0 minor:98 fsType:overlay blockSize:0}] Oct 30 19:48:03.668297 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.667828 6411 manager.go:217] Machine: {Timestamp:2026-10-30 19:48:03.66733371 +0000 UTC m=+0.067505707 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3089876 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e8f99478f83036e32bd3e5bd4a737 SystemUUID:ec2e8f99-478f-8303-6e32-bd3e5bd4a737 BootID:3c546d73-e57f-4c4b-8c7b-a16667c8f43a Filesystems:[{Device:overlay_0-91 DeviceMajor:0 DeviceMinor:91 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:366 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/62561598-09a8-4cb2-b04d-81ce7c0b8411/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:382 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d70ff94cd072c48e42ad24591d42f075373f1326410d83b4e8b19d99a659c1c1/userdata/shm DeviceMajor:0 DeviceMinor:100 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:257 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-409 DeviceMajor:0 DeviceMinor:409 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-446 DeviceMajor:0 DeviceMinor:446 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e1c56ad3d2439ca91bfaa5f7cec666ec8973391cf901b1f202f92fbbd3af7ab2/userdata/shm DeviceMajor:0 DeviceMinor:391 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-395 DeviceMajor:0 DeviceMinor:395 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-397 DeviceMajor:0 DeviceMinor:397 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volumes/kubernetes.io~projected/kube-api-access-fzn4q DeviceMajor:0 DeviceMinor:62 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:288 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:365 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5bbe69d136aefa4fd0c16be97fc6a022036f7086640c1a70cbcf76a8bc98f030/userdata/shm DeviceMajor:0 DeviceMinor:376 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-414 DeviceMajor:0 DeviceMinor:414 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b6d4cafc1eb47eab5e59ca7ae4f78a7d63dd9850f52f470fa55a6d9aa65185ce/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e/volumes/kubernetes.io~projected/kube-api-access-s49fn DeviceMajor:0 DeviceMinor:63 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/27586708-df16-4c22-9b2c-eb87fc94595d/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:203 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~projected/kube-api-access-qdxr6 DeviceMajor:0 DeviceMinor:283 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:293 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/979ecd3c5b31a7cde5cc776ac33c2cfa5c47782725662283a7f08baa058fc829/userdata/shm DeviceMajor:0 DeviceMinor:85 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4348d980cbcc3ed0b92fd34a2facffc6d4a6318e67cc2425df24d79218a2dc0a/userdata/shm DeviceMajor:0 DeviceMinor:204 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:246 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:312 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-333 DeviceMajor:0 DeviceMinor:333 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~projected/kube-api-access-glwp6 DeviceMajor:0 DeviceMinor:375 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/84cf4865b87984598fe3b16ba807871a01d29dd831fb876a67a26dd750d45fa6/userdata/shm DeviceMajor:0 DeviceMinor:253 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:296 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:358 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4b96e6b037426a85d3a891398d5ccfa339192e04ce18ad4402cb8f4a9b6c81bd/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2d70e83de570fe12631bdfc08c7f42ccef4b37586478f4045968742581a1bc44/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46a18c80-179d-46ba-85aa-1489814be80e/volumes/kubernetes.io~projected/kube-api-access-2g5bf DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/38efa6c6146cff0df453fce2e0de42b581de3b4cb80b73e6c2279da08ddeec2d/userdata/shm DeviceMajor:0 DeviceMinor:260 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8498a05bd0dea1a76f393f6262913bac53039ce037a8ac8c03682da8af01858a/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/27586708-df16-4c22-9b2c-eb87fc94595d/volumes/kubernetes.io~projected/kube-api-access-sdvhz DeviceMajor:0 DeviceMinor:208 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:291 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:292 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:367 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-441 DeviceMajor:0 DeviceMinor:441 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3009ff95-7457-4856-be99-e3891d923002/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~projected/kube-api-access-4z65z DeviceMajor:0 DeviceMinor:259 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:277 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:251 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b932d03caaef8b6a30aba82ea26989b0a0e4f4c7849e38bfe5671eb540886c72/userdata/shm DeviceMajor:0 DeviceMinor:369 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:373 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-399 DeviceMajor:0 DeviceMinor:399 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:367 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:360 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/66c35a3180c32bebd160df0dfb67293ffaac1bb6aea0e22cd499740e35bad0d4/userdata/shm DeviceMajor:0 DeviceMinor:89 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/65d4fb96-8b10-487d-9912-bd71dd507120/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:282 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~projected/kube-api-access-wzrnv DeviceMajor:0 DeviceMinor:368 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:374 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c3ae80f992f83041d46b23c1ad5bd0cc8a3ce485ad9d199b83d0e12f8da4ccc4/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:353 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/da64e35d-dc35-4bc8-89cb-23a7eaf75836/volumes/kubernetes.io~projected/kube-api-access-wdmfq DeviceMajor:0 DeviceMinor:388 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/298e0d65-779b-4fd0-8931-43596a15d5fa/volumes/kubernetes.io~projected/kube-api-access-grrnx DeviceMajor:0 DeviceMinor:67 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-184 DeviceMajor:0 DeviceMinor:184 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46a18c80-179d-46ba-85aa-1489814be80e/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/63ee5421345e41a46e0021a3009c43bbd3e4643cbad13bd2386f977af175b314/userdata/shm DeviceMajor:0 DeviceMinor:393 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-403 DeviceMajor:0 DeviceMinor:403 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1742198c1994027a8d5414071eef86a8e5ca5784dcdc1cad0a3052620a232b23/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:295 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~projected/kube-api-access-pmc7x DeviceMajor:0 DeviceMinor:297 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:314 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:359 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1873229-3e23-4ff6-9f57-f8520e9d9852/volumes/kubernetes.io~projected/kube-api-access-t4w2g DeviceMajor:0 DeviceMinor:252 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:294 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-378 DeviceMajor:0 DeviceMinor:378 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/725f1a5800106db465ded6b84f4c801a845e2c9681d0bc801cdab9aae29c51b1/userdata/shm DeviceMajor:0 DeviceMinor:389 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-179 DeviceMajor:0 DeviceMinor:179 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~projected/kube-api-access-n8lff DeviceMajor:0 DeviceMinor:315 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-420 DeviceMajor:0 DeviceMinor:420 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:364 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/410c008ae2e899f7593d50532d68df1c269ac4d3982eb1c761d236dbf79cee6d/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e1e0c2b5-3116-4963-b1df-fef06ea58937/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/108e4ff9a325e8336bd5555eff508483b2a9634189a3084567c4e224fd0bb00d/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:296 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3864b62c-1c37-4c73-89c5-136a1f402078/volumes/kubernetes.io~projected/kube-api-access-hm4wj DeviceMajor:0 DeviceMinor:61 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d03e243eafcb3ce5236c92e0eb4fe70065a94c06f91e1cba3ea5b8be5a092566/userdata/shm DeviceMajor:0 DeviceMinor:305 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/272abb5096c53f8c612c69d6d4490663181c3e3ac2359d3f89490ec57116fb67/userdata/shm DeviceMajor:0 DeviceMinor:316 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:292 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:363 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-436 DeviceMajor:0 DeviceMinor:436 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:296 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57/volumes/kubernetes.io~projected/kube-api-access-f4w9j DeviceMajor:0 DeviceMinor:60 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/62561598-09a8-4cb2-b04d-81ce7c0b8411/volumes/kubernetes.io~projected/kube-api-access-nbgzs DeviceMajor:0 DeviceMinor:65 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f2f79ed46b081f577502da86affe4b5f13e7c71a6535249600fba0757a9204e2/userdata/shm DeviceMajor:0 DeviceMinor:284 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:361 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:362 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/67aca29c1952f4205820a503a15429729f5c93cee8bfbb5ca5e638d824be5ebc/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/65d4fb96-8b10-487d-9912-bd71dd507120/volumes/kubernetes.io~projected/kube-api-access-c7qdn DeviceMajor:0 DeviceMinor:202 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:289 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a6f79c4a-177a-4512-b817-92404fd99b41/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:313 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-371 DeviceMajor:0 DeviceMinor:371 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:367 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-427 DeviceMajor:0 DeviceMinor:427 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc3812fa-69b5-4e6e-bca9-91a3b191ed32/volumes/kubernetes.io~projected/kube-api-access-vkzlx DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/018f82d1-73aa-4c50-bce5-6fef86b947b0/volumes/kubernetes.io~projected/kube-api-access-sm7d6 DeviceMajor:0 DeviceMinor:66 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:290 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:296 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:258 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a3757b61-1b91-4739-a7fc-ecbfcb4feea6/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:387 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bb90a295-9822-4874-9014-0a91ac7a73c5/volumes/kubernetes.io~projected/kube-api-access-kpw8x DeviceMajor:0 DeviceMinor:64 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:367 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:272abb5096c53f8 MacAddress:9e:56:29:f2:6f:d8 Speed:10000 Mtu:8901} {Name:38efa6c6146cff0 MacAddress:aa:a6:0d:df:80:5c Speed:10000 Mtu:8901} {Name:4348d980cbcc3ed MacAddress:b6:9e:0c:7b:b1:42 Speed:10000 Mtu:8901} {Name:5bbe69d136aefa4 MacAddress:c2:7c:57:1e:c0:ae Speed:10000 Mtu:8901} {Name:63ee5421345e41a MacAddress:96:01:6a:15:4a:44 Speed:10000 Mtu:8901} {Name:725f1a5800106db MacAddress:9e:42:7b:b7:20:ec Speed:10000 Mtu:8901} {Name:8498a05bd0dea1a MacAddress:6a:03:c1:94:b0:41 Speed:10000 Mtu:8901} {Name:84cf4865b879845 MacAddress:12:34:0d:52:ce:2c Speed:10000 Mtu:8901} {Name:b932d03caaef8b6 MacAddress:c2:ca:b4:91:4b:e4 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:e4:0b:9c:05 Speed:0 Mtu:9001} {Name:br-int MacAddress:b6:51:84:db:ae:b4 Speed:0 Mtu:8901} {Name:c3ae80f992f8304 MacAddress:8a:8e:c6:71:1e:8a Speed:10000 Mtu:8901} {Name:d03e243eafcb3ce MacAddress:fa:88:3d:4c:b2:9e Speed:10000 Mtu:8901} {Name:e1c56ad3d2439ca MacAddress:62:10:4a:71:05:57 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:e4:0b:9c:05 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:b6:e1:1d:5e:55:b7 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:5e:d0:25:43:53:0e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 19:48:03.668297 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.668302 6411 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 19:48:03.668638 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.668361 6411 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 19:48:03.668744 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.668715 6411 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 19:48:03.668966 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.668747 6411 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 19:48:03.669022 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.668971 6411 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 19:48:03.669022 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.668984 6411 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 19:48:03.669022 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.668999 6411 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 19:48:03.669022 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669018 6411 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 19:48:03.669167 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669158 6411 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 19:48:03.669396 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669385 6411 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 19:48:03.669550 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669539 6411 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 19:48:03.669596 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669587 6411 kubelet.go:493] "Attempting to sync node with API server" Oct 30 19:48:03.669627 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669603 6411 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 19:48:03.669627 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669617 6411 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 19:48:03.669706 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669630 6411 kubelet.go:405] "Adding apiserver pod source" Oct 30 19:48:03.669706 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.669641 6411 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 19:48:03.670366 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.670350 6411 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 19:48:03.670366 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.670364 6411 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 19:48:03.671093 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.671075 6411 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 19:48:03.673980 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.673966 6411 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 19:48:03.673980 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.673979 6411 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 19:48:03.674100 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674080 6411 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 19:48:03.674287 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674275 6411 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 19:48:03.674321 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674311 6411 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 19:48:03.674434 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674416 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 19:48:03.674434 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674432 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 19:48:03.674434 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674438 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674445 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674450 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674457 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674466 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674474 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674482 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674488 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674498 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674507 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674534 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 19:48:03.674653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.674541 6411 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 19:48:03.678173 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.678154 6411 server.go:1263] "Started kubelet" Oct 30 19:48:03.678301 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.678251 6411 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 19:48:03.678359 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.678323 6411 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 19:48:03.678666 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.678607 6411 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 19:48:03.678696 ip-10-0-142-168 systemd[1]: Started Kubernetes Kubelet. Oct 30 19:48:03.679244 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.678677 6411 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 19:48:03.680583 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.680556 6411 server.go:317] "Adding debug handlers to kubelet server" Oct 30 19:48:03.686731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.684513 6411 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 19:48:03.686731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.684974 6411 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 19:48:03.686731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.685117 6411 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:41:52 +0000 UTC" deadline="2028-08-10 00:15:05.886816766 +0000 UTC" Oct 30 19:48:03.686731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.685151 6411 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15580h27m2.201669257s" Oct 30 19:48:03.686731 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:03.685199 6411 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-168.ec2.internal\" not found" Oct 30 19:48:03.686731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.685259 6411 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 19:48:03.686731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.685267 6411 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 19:48:03.686731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.685391 6411 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 19:48:03.688756 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688733 6411 factory.go:55] Registering systemd factory Oct 30 19:48:03.688756 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688756 6411 factory.go:223] Registration of the systemd container factory successfully Oct 30 19:48:03.688935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688837 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46a18c80-179d-46ba-85aa-1489814be80e" volumeName="kubernetes.io/empty-dir/46a18c80-179d-46ba-85aa-1489814be80e-tmp-dir" seLinuxMountContext="" Oct 30 19:48:03.688935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688862 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46a18c80-179d-46ba-85aa-1489814be80e" volumeName="kubernetes.io/projected/46a18c80-179d-46ba-85aa-1489814be80e-kube-api-access-2g5bf" seLinuxMountContext="" Oct 30 19:48:03.688935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688875 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config" seLinuxMountContext="" Oct 30 19:48:03.688935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688888 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b569a9c0-5370-4ceb-9a82-90dad326ffad" volumeName="kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config" seLinuxMountContext="" Oct 30 19:48:03.688935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688926 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ba49ca5-7b3e-42bc-aa61-59004aeb73dc" volumeName="kubernetes.io/empty-dir/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-textfile" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688939 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688952 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688965 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688981 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1873229-3e23-4ff6-9f57-f8520e9d9852" volumeName="kubernetes.io/projected/c1873229-3e23-4ff6-9f57-f8520e9d9852-kube-api-access-t4w2g" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.688993 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1e0c2b5-3116-4963-b1df-fef06ea58937" volumeName="kubernetes.io/secret/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689006 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46a18c80-179d-46ba-85aa-1489814be80e" volumeName="kubernetes.io/secret/46a18c80-179d-46ba-85aa-1489814be80e-metrics-tls" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689041 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689050 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689057 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689067 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65d4fb96-8b10-487d-9912-bd71dd507120" volumeName="kubernetes.io/secret/65d4fb96-8b10-487d-9912-bd71dd507120-cert" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689080 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27586708-df16-4c22-9b2c-eb87fc94595d" volumeName="kubernetes.io/configmap/27586708-df16-4c22-9b2c-eb87fc94595d-kube-rbac-proxy-cm" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689087 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42a9532b-3f87-4fac-816a-d293b921ce5c" volumeName="kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689094 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42a9532b-3f87-4fac-816a-d293b921ce5c" volumeName="kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689101 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689108 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689116 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62561598-09a8-4cb2-b04d-81ce7c0b8411" volumeName="kubernetes.io/projected/62561598-09a8-4cb2-b04d-81ce7c0b8411-kube-api-access-nbgzs" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689124 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689131 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b569a9c0-5370-4ceb-9a82-90dad326ffad" volumeName="kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689138 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689145 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689151 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6f79c4a-177a-4512-b817-92404fd99b41" volumeName="kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-client-certs" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689158 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb90a295-9822-4874-9014-0a91ac7a73c5" volumeName="kubernetes.io/projected/bb90a295-9822-4874-9014-0a91ac7a73c5-kube-api-access-kpw8x" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689166 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1873229-3e23-4ff6-9f57-f8520e9d9852" volumeName="kubernetes.io/configmap/c1873229-3e23-4ff6-9f57-f8520e9d9852-metrics-client-ca" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689177 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="da64e35d-dc35-4bc8-89cb-23a7eaf75836" volumeName="kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689184 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc3812fa-69b5-4e6e-bca9-91a3b191ed32" volumeName="kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-tuned" seLinuxMountContext="" Oct 30 19:48:03.689180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689191 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="018f82d1-73aa-4c50-bce5-6fef86b947b0" volumeName="kubernetes.io/projected/018f82d1-73aa-4c50-bce5-6fef86b947b0-kube-api-access-sm7d6" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689204 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298e0d65-779b-4fd0-8931-43596a15d5fa" volumeName="kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-binary-copy" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689217 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42a9532b-3f87-4fac-816a-d293b921ce5c" volumeName="kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689225 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689232 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689239 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689247 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f8d4383-c954-4fed-aadf-f1ee7c0eb24e" volumeName="kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-daemon-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689253 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1873229-3e23-4ff6-9f57-f8520e9d9852" volumeName="kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-tls" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689260 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46a18c80-179d-46ba-85aa-1489814be80e" volumeName="kubernetes.io/configmap/46a18c80-179d-46ba-85aa-1489814be80e-config-volume" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689267 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689275 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62561598-09a8-4cb2-b04d-81ce7c0b8411" volumeName="kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689286 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6f79c4a-177a-4512-b817-92404fd99b41" volumeName="kubernetes.io/projected/a6f79c4a-177a-4512-b817-92404fd99b41-kube-api-access-n8lff" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689293 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b569a9c0-5370-4ceb-9a82-90dad326ffad" volumeName="kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689300 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1e0c2b5-3116-4963-b1df-fef06ea58937" volumeName="kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-script-lib" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689309 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1e0c2b5-3116-4963-b1df-fef06ea58937" volumeName="kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689319 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1e0c2b5-3116-4963-b1df-fef06ea58937" volumeName="kubernetes.io/projected/e1e0c2b5-3116-4963-b1df-fef06ea58937-kube-api-access-fzn4q" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689327 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ba49ca5-7b3e-42bc-aa61-59004aeb73dc" volumeName="kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689334 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42a9532b-3f87-4fac-816a-d293b921ce5c" volumeName="kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689341 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65d4fb96-8b10-487d-9912-bd71dd507120" volumeName="kubernetes.io/projected/65d4fb96-8b10-487d-9912-bd71dd507120-kube-api-access-c7qdn" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689348 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb90a295-9822-4874-9014-0a91ac7a73c5" volumeName="kubernetes.io/empty-dir/bb90a295-9822-4874-9014-0a91ac7a73c5-tmp-dir" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689356 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ba49ca5-7b3e-42bc-aa61-59004aeb73dc" volumeName="kubernetes.io/projected/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-kube-api-access-qdxr6" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689363 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ba49ca5-7b3e-42bc-aa61-59004aeb73dc" volumeName="kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-tls" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689370 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3009ff95-7457-4856-be99-e3891d923002" volumeName="kubernetes.io/configmap/3009ff95-7457-4856-be99-e3891d923002-konnectivity-ca" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689378 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3009ff95-7457-4856-be99-e3891d923002" volumeName="kubernetes.io/secret/3009ff95-7457-4856-be99-e3891d923002-agent-certs" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689385 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689391 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689405 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689412 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689419 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42a9532b-3f87-4fac-816a-d293b921ce5c" volumeName="kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689427 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689478 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689488 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6f79c4a-177a-4512-b817-92404fd99b41" volumeName="kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689870 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b569a9c0-5370-4ceb-9a82-90dad326ffad" volumeName="kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689877 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc3812fa-69b5-4e6e-bca9-91a3b191ed32" volumeName="kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-tmp" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689885 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27586708-df16-4c22-9b2c-eb87fc94595d" volumeName="kubernetes.io/projected/27586708-df16-4c22-9b2c-eb87fc94595d-kube-api-access-sdvhz" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689907 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27586708-df16-4c22-9b2c-eb87fc94595d" volumeName="kubernetes.io/secret/27586708-df16-4c22-9b2c-eb87fc94595d-insights-runtime-extractor-tls" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689917 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298e0d65-779b-4fd0-8931-43596a15d5fa" volumeName="kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689925 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298e0d65-779b-4fd0-8931-43596a15d5fa" volumeName="kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689934 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3864b62c-1c37-4c73-89c5-136a1f402078" volumeName="kubernetes.io/configmap/3864b62c-1c37-4c73-89c5-136a1f402078-iptables-alerter-script" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689942 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42a9532b-3f87-4fac-816a-d293b921ce5c" volumeName="kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689949 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42a9532b-3f87-4fac-816a-d293b921ce5c" volumeName="kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689957 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689964 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27586708-df16-4c22-9b2c-eb87fc94595d" volumeName="kubernetes.io/empty-dir/27586708-df16-4c22-9b2c-eb87fc94595d-data-volume" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689971 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42a9532b-3f87-4fac-816a-d293b921ce5c" volumeName="kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689980 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.689987 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690012 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3757b61-1b91-4739-a7fc-ecbfcb4feea6" volumeName="kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690021 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6f79c4a-177a-4512-b817-92404fd99b41" volumeName="kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-metrics-server-audit-profiles" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690030 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6f79c4a-177a-4512-b817-92404fd99b41" volumeName="kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-client-ca-bundle" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690038 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b569a9c0-5370-4ceb-9a82-90dad326ffad" volumeName="kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690046 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690054 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690061 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690068 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690076 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690083 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b569a9c0-5370-4ceb-9a82-90dad326ffad" volumeName="kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690090 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57" volumeName="kubernetes.io/configmap/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-serviceca" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690098 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57" volumeName="kubernetes.io/projected/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-kube-api-access-f4w9j" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690105 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298e0d65-779b-4fd0-8931-43596a15d5fa" volumeName="kubernetes.io/projected/298e0d65-779b-4fd0-8931-43596a15d5fa-kube-api-access-grrnx" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690114 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3864b62c-1c37-4c73-89c5-136a1f402078" volumeName="kubernetes.io/projected/3864b62c-1c37-4c73-89c5-136a1f402078-kube-api-access-hm4wj" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690121 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690128 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f8d4383-c954-4fed-aadf-f1ee7c0eb24e" volumeName="kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cni-binary-copy" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690135 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b569a9c0-5370-4ceb-9a82-90dad326ffad" volumeName="kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690143 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1873229-3e23-4ff6-9f57-f8520e9d9852" volumeName="kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690150 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc3812fa-69b5-4e6e-bca9-91a3b191ed32" volumeName="kubernetes.io/projected/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-kube-api-access-vkzlx" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690159 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ba49ca5-7b3e-42bc-aa61-59004aeb73dc" volumeName="kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-metrics-client-ca" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690166 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55f5f63c-a55a-4f7c-858c-bb3051ef73be" volumeName="kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690173 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6f79c4a-177a-4512-b817-92404fd99b41" volumeName="kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-tls" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690182 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ba49ca5-7b3e-42bc-aa61-59004aeb73dc" volumeName="kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-accelerators-collector-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690190 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a61eb1f-e59d-4050-b108-20b6408b819a" volumeName="kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690197 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f8d4383-c954-4fed-aadf-f1ee7c0eb24e" volumeName="kubernetes.io/projected/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-kube-api-access-s49fn" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690205 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6f79c4a-177a-4512-b817-92404fd99b41" volumeName="kubernetes.io/empty-dir/a6f79c4a-177a-4512-b817-92404fd99b41-audit-log" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690212 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b569a9c0-5370-4ceb-9a82-90dad326ffad" volumeName="kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690219 6411 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1e0c2b5-3116-4963-b1df-fef06ea58937" volumeName="kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-env-overrides" seLinuxMountContext="" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690226 6411 reconstruct.go:97] "Volume reconstruction finished" Oct 30 19:48:03.690363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.690230 6411 reconciler.go:29] "Reconciler: start to sync state" Oct 30 19:48:03.694120 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.692319 6411 factory.go:153] Registering CRI-O factory Oct 30 19:48:03.694120 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.692382 6411 factory.go:223] Registration of the crio container factory successfully Oct 30 19:48:03.694120 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.692634 6411 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 19:48:03.694120 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.692656 6411 factory.go:103] Registering Raw factory Oct 30 19:48:03.694120 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.692670 6411 manager.go:1199] Started watching for new ooms in manager Oct 30 19:48:03.694120 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.693062 6411 manager.go:319] Starting recovery of all containers Oct 30 19:48:03.724088 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.724031 6411 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 19:48:03.725532 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.725479 6411 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 19:48:03.725532 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.725506 6411 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 19:48:03.725532 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.725529 6411 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 19:48:03.725776 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:03.725572 6411 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 19:48:03.739720 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.739688 6411 generic.go:356] "Generic (PLEG): container finished" podID="0ba49ca5-7b3e-42bc-aa61-59004aeb73dc" containerID="9105012d760598cccaab9c757aa6fbc0a1b7f3a89fabbfc9f0c180cf21fdfeda" exitCode=0 Oct 30 19:48:03.740801 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.740778 6411 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 19:48:03.742003 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.741746 6411 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:48:03.742003 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.741864 6411 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:48:03.754972 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.754948 6411 generic.go:356] "Generic (PLEG): container finished" podID="22f1ffa9a97654c51d3e03111cf4f982" containerID="93fd4701257dcad2a38712b9fa5b55ac6b15e96fba5f350ed25089b16b851392" exitCode=0 Oct 30 19:48:03.765172 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.765148 6411 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:48:03.765258 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.765219 6411 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:48:03.777441 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.777417 6411 generic.go:356] "Generic (PLEG): container finished" podID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" exitCode=0 Oct 30 19:48:03.784782 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.784747 6411 generic.go:356] "Generic (PLEG): container finished" podID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerID="9503fd4b4d23f77269e7bdb0f71b093fe36d61208fc9c169139ba6cd9d316a2c" exitCode=0 Oct 30 19:48:03.790413 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.790387 6411 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="4eaa7f25ed27b0beef4e2d0783eac63ede6069d11382f3c6253ed953d5719ebb" exitCode=0 Oct 30 19:48:03.790500 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.790418 6411 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="c71c60203c2803419b7a8b72d087e947d0b64ae545eca047f0e1caed485d257a" exitCode=0 Oct 30 19:48:03.790500 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.790431 6411 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="005a477cd8cc9fefcaf2305e84d74c932acfd9e09c0c9fba24fc0c5a07fed6f6" exitCode=0 Oct 30 19:48:03.790500 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.790441 6411 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="59c9942842595a1405b327adb0eddb434af665e9de26dfdd45b0dcf00a583236" exitCode=0 Oct 30 19:48:03.790500 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.790448 6411 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="81704250eeca65fe674598dc5d1b23d2233cf2ab680ec669895eb806f8b25a5a" exitCode=0 Oct 30 19:48:03.790500 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.790456 6411 generic.go:356] "Generic (PLEG): container finished" podID="298e0d65-779b-4fd0-8931-43596a15d5fa" containerID="048e2738ddedbdfe5c782a354b92f182b5880d0a0e639a6c998eaa9b5911d5ff" exitCode=0 Oct 30 19:48:03.812792 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.812772 6411 manager.go:324] Recovery completed Oct 30 19:48:03.826553 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:03.826525 6411 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 19:48:03.837174 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837155 6411 cpu_manager.go:226] "Starting" policy="none" Oct 30 19:48:03.837174 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837170 6411 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 19:48:03.837318 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837189 6411 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 19:48:03.837370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837356 6411 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 19:48:03.837498 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837372 6411 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 19:48:03.837498 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837398 6411 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 19:48:03.837498 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837406 6411 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 19:48:03.837498 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837413 6411 policy_none.go:50] "Start" Oct 30 19:48:03.837498 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837426 6411 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 19:48:03.837498 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837439 6411 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 19:48:03.837769 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837620 6411 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 19:48:03.837769 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837632 6411 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 19:48:03.837769 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.837640 6411 policy_none.go:44] "Start" Oct 30 19:48:03.842330 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.842302 6411 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 19:48:03.845026 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.845012 6411 manager.go:341] "Starting Device Plugin manager" Oct 30 19:48:03.845578 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:03.845037 6411 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 19:48:03.845578 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.845578 6411 server.go:87] "Starting device plugin registration server" Oct 30 19:48:03.845833 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.845819 6411 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 19:48:03.845941 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.845831 6411 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 19:48:03.845981 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.845939 6411 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 19:48:03.846076 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.846037 6411 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 19:48:03.846151 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.846101 6411 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 19:48:03.846151 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.846109 6411 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 19:48:03.846305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.846206 6411 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:48:03.846066654Z","UUID":"2cd82ad5-d445-4b6c-956e-f7b934dab02c","Handler":null,"Name":"","Endpoint":""} Oct 30 19:48:03.847642 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:03.847623 6411 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 19:48:03.847729 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.847645 6411 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 19:48:03.847729 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.847675 6411 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 19:48:03.946499 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.946426 6411 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:48:03.948555 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.948529 6411 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientMemory" Oct 30 19:48:03.948668 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.948566 6411 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 19:48:03.948668 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.948581 6411 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-168.ec2.internal" event="NodeHasSufficientPID" Oct 30 19:48:03.948668 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.948629 6411 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-168.ec2.internal" Oct 30 19:48:03.995157 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.995129 6411 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-168.ec2.internal" Oct 30 19:48:03.995296 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:03.995204 6411 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.027256 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.027203 6411 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal"] Oct 30 19:48:04.027792 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.027758 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" event={"ID":"22f1ffa9a97654c51d3e03111cf4f982","Type":"ContainerStarted","Data":"8883777d502bdb5c79032cc3a3abe5bef45bada191ce3491fe3bd5e203c5d83b"} Oct 30 19:48:04.027947 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.027800 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" event={"ID":"22f1ffa9a97654c51d3e03111cf4f982","Type":"ContainerDied","Data":"93fd4701257dcad2a38712b9fa5b55ac6b15e96fba5f350ed25089b16b851392"} Oct 30 19:48:04.027947 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.027820 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" event={"ID":"22f1ffa9a97654c51d3e03111cf4f982","Type":"ContainerStarted","Data":"410c008ae2e899f7593d50532d68df1c269ac4d3982eb1c761d236dbf79cee6d"} Oct 30 19:48:04.027947 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.027860 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" event={"ID":"41493c3824738811979a28105df74e81","Type":"ContainerStarted","Data":"a5706c86f4ee1ac55bfeec1a16cdd4d6ae2fbfaa77505814e417aa98c997eb61"} Oct 30 19:48:04.027947 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.027880 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" event={"ID":"41493c3824738811979a28105df74e81","Type":"ContainerStarted","Data":"b6d4cafc1eb47eab5e59ca7ae4f78a7d63dd9850f52f470fa55a6d9aa65185ce"} Oct 30 19:48:04.027947 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.027862 6411 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.028123 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.027956 6411 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.064893 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.064873 6411 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:48:04.064999 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:04.064945 6411 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-168.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.066117 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.066106 6411 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:48:04.066156 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:04.066130 6411 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.142466 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.142443 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.142578 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.142469 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41493c3824738811979a28105df74e81-config\") pod \"kube-apiserver-proxy-ip-10-0-142-168.ec2.internal\" (UID: \"41493c3824738811979a28105df74e81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.142578 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.142487 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.243080 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.243021 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.243080 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.243061 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.243204 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.243086 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41493c3824738811979a28105df74e81-config\") pod \"kube-apiserver-proxy-ip-10-0-142-168.ec2.internal\" (UID: \"41493c3824738811979a28105df74e81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.243204 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.243091 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.243204 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.243128 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41493c3824738811979a28105df74e81-config\") pod \"kube-apiserver-proxy-ip-10-0-142-168.ec2.internal\" (UID: \"41493c3824738811979a28105df74e81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.243204 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.243148 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22f1ffa9a97654c51d3e03111cf4f982-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" (UID: \"22f1ffa9a97654c51d3e03111cf4f982\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.670629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.670599 6411 apiserver.go:52] "Watching apiserver" Oct 30 19:48:04.715384 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.715356 6411 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 19:48:04.716034 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.716008 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-p6pc9","openshift-image-registry/node-ca-ghl6s","openshift-insights/insights-runtime-extractor-887vm","openshift-monitoring/alertmanager-main-0","openshift-network-diagnostics/network-check-target-bvdgb","openshift-multus/multus-additional-cni-plugins-gnmqz","openshift-network-operator/iptables-alerter-9tnc4","openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal","openshift-monitoring/node-exporter-dcpkx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd","openshift-multus/multus-lnxz5","kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal","openshift-dns/dns-default-cwb2s","openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w","openshift-console/console-57b59b9797-ptgs8","kube-system/konnectivity-agent-44glf","openshift-console/console-6f66d775b6-dq96j","openshift-monitoring/prometheus-k8s-0","openshift-cluster-node-tuning-operator/tuned-f2tdf","kube-system/global-pull-secret-syncer-jvsfx","openshift-multus/network-metrics-daemon-lb8sw","openshift-ovn-kubernetes/ovnkube-node-x62cc","openshift-ingress-canary/ingress-canary-f8s4w"] Oct 30 19:48:04.736005 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.735977 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.736118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.735986 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.736167 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.736151 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.736773 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.736723 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 19:48:04.736885 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.736789 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.736954 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.736892 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 19:48:04.737145 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737126 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 19:48:04.737208 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737189 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.737280 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737260 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-97cb9\"" Oct 30 19:48:04.737323 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737275 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mbspd\"" Oct 30 19:48:04.737370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737337 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 19:48:04.737412 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737397 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d5xr8\"" Oct 30 19:48:04.737460 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737125 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.737505 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737457 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.737505 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737346 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 19:48:04.737591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737505 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.737591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737533 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 19:48:04.737591 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737576 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cscqg\"" Oct 30 19:48:04.737719 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737599 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2gzs4\"" Oct 30 19:48:04.737719 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.737638 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.744753 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.744727 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-grfkh\"" Oct 30 19:48:04.745005 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.744986 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wnngx\"" Oct 30 19:48:04.745060 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.744986 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n5k26\"" Oct 30 19:48:04.745484 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.745469 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 19:48:04.745733 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.745713 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.745768 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.745741 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 19:48:04.745768 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.745746 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.745768 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.745748 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 19:48:04.745855 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.745778 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 19:48:04.745951 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.745940 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.746735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.746723 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 19:48:04.747004 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.746993 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 19:48:04.747043 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.747017 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.747075 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.747045 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 19:48:04.749839 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.749827 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.749881 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.749832 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 19:48:04.753482 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.753469 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.756165 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.756153 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.756451 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.756432 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 19:48:04.756528 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.756472 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 19:48:04.756582 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.756483 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 19:48:04.756582 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.756485 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 19:48:04.756582 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.756501 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.756800 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.756786 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.757079 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757063 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.757214 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757078 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 19:48:04.757320 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757288 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 19:48:04.757381 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757328 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 19:48:04.757381 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757339 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 19:48:04.757523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757442 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qlg5x\"" Oct 30 19:48:04.757523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757468 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l5g68\"" Oct 30 19:48:04.757523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757480 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-h4pgj\"" Oct 30 19:48:04.757523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757491 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-bvhff\"" Oct 30 19:48:04.757523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757503 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 19:48:04.757523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757516 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 19:48:04.757523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757543 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 19:48:04.757523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757549 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wgtz9\"" Oct 30 19:48:04.758006 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757596 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jnc65\"" Oct 30 19:48:04.758006 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757627 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7g4j\"" Oct 30 19:48:04.758006 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757673 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9uipjplehrdud\"" Oct 30 19:48:04.758006 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757699 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k6b6j\"" Oct 30 19:48:04.758006 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757708 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 19:48:04.758006 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.757717 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-67dw4\"" Oct 30 19:48:04.758650 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.758637 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 19:48:04.758748 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.758652 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 19:48:04.758748 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.758690 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 19:48:04.758984 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.758970 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 19:48:04.770622 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.770606 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 19:48:04.771109 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.771096 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 19:48:04.771169 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.771153 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 19:48:04.788018 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.788001 6411 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 19:48:04.794172 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.794155 6411 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.794303 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.794291 6411 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.805344 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.805320 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:04.819749 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.819729 6411 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:48:04.819836 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:04.819772 6411 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.836755 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.836731 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 19:48:04.836755 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.836735 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.837036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837019 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 19:48:04.837112 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837096 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 19:48:04.837167 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837154 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 19:48:04.837213 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837197 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.837289 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837277 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.837340 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837276 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:48:04.837340 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837311 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 19:48:04.837425 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837347 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 19:48:04.837425 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837370 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 19:48:04.837538 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.837525 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 19:48:04.838462 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.838447 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 19:48:04.838523 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.838475 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 19:48:04.838855 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.838833 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 19:48:04.842166 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.842148 6411 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:48:04.842266 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:04.842189 6411 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-168.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-168.ec2.internal" Oct 30 19:48:04.842956 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.842934 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 19:48:04.843198 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843177 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 19:48:04.843462 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843446 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 19:48:04.843579 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843558 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2llhk\"" Oct 30 19:48:04.843709 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843685 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 30 19:48:04.843815 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843801 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7krm2j7bnghi9\"" Oct 30 19:48:04.843947 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843927 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 19:48:04.844101 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844082 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 19:48:04.844171 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844152 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 19:48:04.844274 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844259 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 19:48:04.844424 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844400 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rgk57\"" Oct 30 19:48:04.844489 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844263 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 19:48:04.844489 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844282 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 19:48:04.844489 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843463 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 19:48:04.844614 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843993 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 19:48:04.844614 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844411 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-h5z6r\"" Oct 30 19:48:04.844614 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844154 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-25nmp\"" Oct 30 19:48:04.844738 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.843803 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 19:48:04.844738 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844638 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 19:48:04.844816 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.844803 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 19:48:04.845117 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845095 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.845179 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845142 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-dbus\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:48:04.845179 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845173 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-systemd-units\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.845278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845202 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-etc-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.845278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845235 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.845278 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845265 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cni-binary-copy\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.845414 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845289 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.845414 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845344 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-ovn\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.845414 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845377 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27586708-df16-4c22-9b2c-eb87fc94595d-data-volume\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.845414 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845407 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27586708-df16-4c22-9b2c-eb87fc94595d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.845585 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845437 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.845585 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845465 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-serviceca\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:48:04.845585 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845489 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.845585 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845524 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-socket-dir-parent\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.845585 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845551 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-host\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:48:04.845585 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845579 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46a18c80-179d-46ba-85aa-1489814be80e-metrics-tls\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:04.845838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845605 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.845838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845636 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-run\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.845838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845664 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46a18c80-179d-46ba-85aa-1489814be80e-tmp-dir\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:04.845838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845694 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.845838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845727 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-metrics-client-ca\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.845838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845757 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.845838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845782 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm4wj\" (UniqueName: \"kubernetes.io/projected/3864b62c-1c37-4c73-89c5-136a1f402078-kube-api-access-hm4wj\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:48:04.845838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845812 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845841 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-multus-certs\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845874 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845924 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb90a295-9822-4874-9014-0a91ac7a73c5-hosts-file\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845950 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-k8s-cni-cncf-io\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.845983 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-client-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846012 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8lff\" (UniqueName: \"kubernetes.io/projected/a6f79c4a-177a-4512-b817-92404fd99b41-kube-api-access-n8lff\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846041 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846074 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-bin\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846104 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z65z\" (UniqueName: \"kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846137 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.846185 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846176 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-kubernetes\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.846640 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846207 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-tuned\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.846640 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846241 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-system-cni-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.846640 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846273 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.846640 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846277 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.846640 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846302 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.846640 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846331 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glwp6\" (UniqueName: \"kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.846640 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846488 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 30 19:48:04.846640 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846583 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.847037 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846647 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-conf\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.847037 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846661 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46a18c80-179d-46ba-85aa-1489814be80e-tmp-dir\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:04.847037 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846685 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cnibin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.847037 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846880 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-client-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.847037 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.846985 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.847257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847226 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-tuned\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.847313 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847266 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-metrics-client-ca\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.847386 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847367 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.847452 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847432 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.847515 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847490 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1873229-3e23-4ff6-9f57-f8520e9d9852-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:48:04.847567 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847520 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46a18c80-179d-46ba-85aa-1489814be80e-metrics-tls\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:04.847651 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847632 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-serviceca\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:48:04.847702 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847641 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27586708-df16-4c22-9b2c-eb87fc94595d-data-volume\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.847702 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847650 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.847806 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847719 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.847806 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847790 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.847923 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847854 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.847923 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847871 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.847923 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847790 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.848097 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847939 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.848097 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847945 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-textfile\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.848097 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847967 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27586708-df16-4c22-9b2c-eb87fc94595d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.848097 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.847983 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.848097 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848018 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-metrics-server-audit-profiles\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.848097 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848016 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-textfile\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.848097 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848091 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.848402 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848142 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-script-lib\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.848402 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848155 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cni-binary-copy\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.848402 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848173 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-original-pull-secret\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:48:04.848402 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848157 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1873229-3e23-4ff6-9f57-f8520e9d9852-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:48:04.848402 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848298 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848401 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848410 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-modprobe-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848437 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-metrics-server-audit-profiles\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848454 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm7d6\" (UniqueName: \"kubernetes.io/projected/018f82d1-73aa-4c50-bce5-6fef86b947b0-kube-api-access-sm7d6\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848491 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grrnx\" (UniqueName: \"kubernetes.io/projected/298e0d65-779b-4fd0-8931-43596a15d5fa-kube-api-access-grrnx\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848521 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a6f79c4a-177a-4512-b817-92404fd99b41-audit-log\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848547 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848559 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848593 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-script-lib\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.848629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848617 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-sys-fs\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848656 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4w9j\" (UniqueName: \"kubernetes.io/projected/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-kube-api-access-f4w9j\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848670 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a6f79c4a-177a-4512-b817-92404fd99b41-audit-log\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848685 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848736 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848735 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-var-lib-kubelet\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848781 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848800 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-system-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848837 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-sys\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.849062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.848879 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-log-socket\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.849339 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849285 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:48:04.849339 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849324 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-host\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.849425 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849351 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-socket-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.849425 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849376 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s49fn\" (UniqueName: \"kubernetes.io/projected/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-kube-api-access-s49fn\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.849502 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849431 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.849547 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849514 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62561598-09a8-4cb2-b04d-81ce7c0b8411-metrics-certs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:48:04.849594 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849552 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.849594 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849575 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.849693 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849626 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.849744 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849727 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.849801 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849756 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb90a295-9822-4874-9014-0a91ac7a73c5-tmp-dir\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:48:04.849801 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849773 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.849801 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849796 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.849965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849816 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.849965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849823 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb90a295-9822-4874-9014-0a91ac7a73c5-tmp-dir\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:48:04.849965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849819 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-kubelet\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.849965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849854 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-cnibin\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.849965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849940 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849969 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-kubelet-config\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.849996 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850017 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-kubelet\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850038 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-config\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850069 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850085 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850119 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65d4fb96-8b10-487d-9912-bd71dd507120-cert\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850146 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.850194 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850176 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850201 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850227 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850253 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-tls\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850279 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-netd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850316 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850253 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovnkube-config\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850345 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65d4fb96-8b10-487d-9912-bd71dd507120-cert\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850419 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850431 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850449 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850432 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850490 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-sys\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850510 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850491 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850524 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-tls\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.850593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850542 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850664 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850687 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3009ff95-7457-4856-be99-e3891d923002-agent-certs\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850689 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850774 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7qdn\" (UniqueName: \"kubernetes.io/projected/65d4fb96-8b10-487d-9912-bd71dd507120-kube-api-access-c7qdn\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850828 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850844 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850862 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850877 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850918 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850951 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdxr6\" (UniqueName: \"kubernetes.io/projected/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-kube-api-access-qdxr6\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850973 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3009ff95-7457-4856-be99-e3891d923002-agent-certs\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.850985 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851040 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851066 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-systemd\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851092 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851094 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-registration-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.851137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851138 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851161 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpw8x\" (UniqueName: \"kubernetes.io/projected/bb90a295-9822-4874-9014-0a91ac7a73c5-kube-api-access-kpw8x\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851188 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-conf-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851217 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851213 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-daemon-config\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851245 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298e0d65-779b-4fd0-8931-43596a15d5fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851255 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-etc-kubernetes\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851288 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-client-certs\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851308 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851325 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-netns\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851356 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851372 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851388 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851402 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-tmp\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851417 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-multus\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851432 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-node-log\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851443 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-daemon-config\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851465 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851493 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-tmp\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851499 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3864b62c-1c37-4c73-89c5-136a1f402078-host-slash\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851518 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851537 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851632 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f79c4a-177a-4512-b817-92404fd99b41-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851650 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a6f79c4a-177a-4512-b817-92404fd99b41-secret-metrics-server-client-certs\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851788 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851819 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-lib-modules\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851852 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-wtmp\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851871 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851871 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851884 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1873229-3e23-4ff6-9f57-f8520e9d9852-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:48:04.851935 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.851921 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852028 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852038 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852057 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852107 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852143 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852145 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-device-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852254 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852263 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852283 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852384 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852386 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852390 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852411 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852456 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-root\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852489 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-slash\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852534 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdvhz\" (UniqueName: \"kubernetes.io/projected/27586708-df16-4c22-9b2c-eb87fc94595d-kube-api-access-sdvhz\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852561 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852563 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysconfig\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852606 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-bin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852638 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4w2g\" (UniqueName: \"kubernetes.io/projected/c1873229-3e23-4ff6-9f57-f8520e9d9852-kube-api-access-t4w2g\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852671 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-var-lib-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852687 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-env-overrides\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852710 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovn-node-metrics-cert\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852814 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1e0c2b5-3116-4963-b1df-fef06ea58937-env-overrides\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852814 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzn4q\" (UniqueName: \"kubernetes.io/projected/e1e0c2b5-3116-4963-b1df-fef06ea58937-kube-api-access-fzn4q\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.852965 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.852971 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1e0c2b5-3116-4963-b1df-fef06ea58937-ovn-node-metrics-cert\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853023 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27586708-df16-4c22-9b2c-eb87fc94595d-crio-socket\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853083 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3009ff95-7457-4856-be99-e3891d923002-konnectivity-ca\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853112 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkzlx\" (UniqueName: \"kubernetes.io/projected/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-kube-api-access-vkzlx\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853203 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-netns\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853218 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3009ff95-7457-4856-be99-e3891d923002-konnectivity-ca\") pod \"konnectivity-agent-44glf\" (UID: \"3009ff95-7457-4856-be99-e3891d923002\") " pod="kube-system/konnectivity-agent-44glf" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853316 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853350 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27586708-df16-4c22-9b2c-eb87fc94595d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853398 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853470 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853497 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3864b62c-1c37-4c73-89c5-136a1f402078-iptables-alerter-script\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853584 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27586708-df16-4c22-9b2c-eb87fc94595d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853584 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46a18c80-179d-46ba-85aa-1489814be80e-config-volume\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853652 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853681 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3864b62c-1c37-4c73-89c5-136a1f402078-iptables-alerter-script\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853688 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g5bf\" (UniqueName: \"kubernetes.io/projected/46a18c80-179d-46ba-85aa-1489814be80e-kube-api-access-2g5bf\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853715 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853766 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853790 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46a18c80-179d-46ba-85aa-1489814be80e-config-volume\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853877 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853921 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-tls\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853979 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.853970 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzrnv\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.854010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854010 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-systemd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854028 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854047 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854104 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-hostroot\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854121 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-tls\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854132 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854156 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854219 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmc7x\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854316 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854330 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbgzs\" (UniqueName: \"kubernetes.io/projected/62561598-09a8-4cb2-b04d-81ce7c0b8411-kube-api-access-nbgzs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854363 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854453 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854623 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.854763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854650 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.855166 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.854924 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.870292 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.870267 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glwp6\" (UniqueName: \"kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6\") pod \"console-6f66d775b6-dq96j\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:04.871162 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.871146 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8lff\" (UniqueName: \"kubernetes.io/projected/a6f79c4a-177a-4512-b817-92404fd99b41-kube-api-access-n8lff\") pod \"metrics-server-5f9d8fd57c-dkv9w\" (UID: \"a6f79c4a-177a-4512-b817-92404fd99b41\") " pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:04.877264 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.877243 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z65z\" (UniqueName: \"kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z\") pod \"console-57b59b9797-ptgs8\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:04.885180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.885162 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4w9j\" (UniqueName: \"kubernetes.io/projected/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-kube-api-access-f4w9j\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:48:04.885248 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.885166 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm4wj\" (UniqueName: \"kubernetes.io/projected/3864b62c-1c37-4c73-89c5-136a1f402078-kube-api-access-hm4wj\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:48:04.886302 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.886276 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkzlx\" (UniqueName: \"kubernetes.io/projected/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-kube-api-access-vkzlx\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.886356 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.886318 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grrnx\" (UniqueName: \"kubernetes.io/projected/298e0d65-779b-4fd0-8931-43596a15d5fa-kube-api-access-grrnx\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.886476 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.886455 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4w2g\" (UniqueName: \"kubernetes.io/projected/c1873229-3e23-4ff6-9f57-f8520e9d9852-kube-api-access-t4w2g\") pod \"prometheus-operator-9cd89b74c-xbkdh\" (UID: \"c1873229-3e23-4ff6-9f57-f8520e9d9852\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xbkdh" Oct 30 19:48:04.886880 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.886860 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s49fn\" (UniqueName: \"kubernetes.io/projected/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-kube-api-access-s49fn\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.887005 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.886992 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdvhz\" (UniqueName: \"kubernetes.io/projected/27586708-df16-4c22-9b2c-eb87fc94595d-kube-api-access-sdvhz\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.887151 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.887136 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7qdn\" (UniqueName: \"kubernetes.io/projected/65d4fb96-8b10-487d-9912-bd71dd507120-kube-api-access-c7qdn\") pod \"ingress-canary-f8s4w\" (UID: \"65d4fb96-8b10-487d-9912-bd71dd507120\") " pod="openshift-ingress-canary/ingress-canary-f8s4w" Oct 30 19:48:04.888526 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.888510 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpw8x\" (UniqueName: \"kubernetes.io/projected/bb90a295-9822-4874-9014-0a91ac7a73c5-kube-api-access-kpw8x\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:48:04.888766 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.888750 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmc7x\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x\") pod \"alertmanager-main-0\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:04.890866 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.890846 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdxr6\" (UniqueName: \"kubernetes.io/projected/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-kube-api-access-qdxr6\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.891473 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.891455 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdmfq\" (UniqueName: \"kubernetes.io/projected/da64e35d-dc35-4bc8-89cb-23a7eaf75836-kube-api-access-wdmfq\") pod \"network-check-target-bvdgb\" (UID: \"da64e35d-dc35-4bc8-89cb-23a7eaf75836\") " pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:48:04.891650 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.891633 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzn4q\" (UniqueName: \"kubernetes.io/projected/e1e0c2b5-3116-4963-b1df-fef06ea58937-kube-api-access-fzn4q\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.892496 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.892479 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbgzs\" (UniqueName: \"kubernetes.io/projected/62561598-09a8-4cb2-b04d-81ce7c0b8411-kube-api-access-nbgzs\") pod \"network-metrics-daemon-lb8sw\" (UID: \"62561598-09a8-4cb2-b04d-81ce7c0b8411\") " pod="openshift-multus/network-metrics-daemon-lb8sw" Oct 30 19:48:04.894266 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.894246 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm7d6\" (UniqueName: \"kubernetes.io/projected/018f82d1-73aa-4c50-bce5-6fef86b947b0-kube-api-access-sm7d6\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.895349 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.895320 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g5bf\" (UniqueName: \"kubernetes.io/projected/46a18c80-179d-46ba-85aa-1489814be80e-kube-api-access-2g5bf\") pod \"dns-default-cwb2s\" (UID: \"46a18c80-179d-46ba-85aa-1489814be80e\") " pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:04.896227 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.896211 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzrnv\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv\") pod \"prometheus-k8s-0\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:04.955188 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955121 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-netns\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955188 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955177 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-multus\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955203 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-node-log\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955218 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3864b62c-1c37-4c73-89c5-136a1f402078-host-slash\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955236 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-lib-modules\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955252 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-wtmp\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955264 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-node-log\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955234 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-netns\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955270 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-device-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955310 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-multus\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955323 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-device-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955328 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3864b62c-1c37-4c73-89c5-136a1f402078-host-slash\") pod \"iptables-alerter-9tnc4\" (UID: \"3864b62c-1c37-4c73-89c5-136a1f402078\") " pod="openshift-network-operator/iptables-alerter-9tnc4" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955354 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-node-exporter-wtmp\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955377 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.955393 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955399 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-lib-modules\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955407 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-root\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955442 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955452 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-slash\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955484 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysconfig\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955487 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-slash\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955485 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-root\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955510 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-bin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955530 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysconfig\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955535 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-var-lib-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955569 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27586708-df16-4c22-9b2c-eb87fc94595d-crio-socket\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955568 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-var-lib-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955595 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-netns\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955609 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27586708-df16-4c22-9b2c-eb87fc94595d-crio-socket\") pod \"insights-runtime-extractor-887vm\" (UID: \"27586708-df16-4c22-9b2c-eb87fc94595d\") " pod="openshift-insights/insights-runtime-extractor-887vm" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955623 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955592 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-cni-bin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955629 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-netns\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955653 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-systemd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955681 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955690 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955705 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-hostroot\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955711 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-systemd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955724 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-dbus\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955732 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955763 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-hostroot\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955774 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-dbus\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955757 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-systemd-units\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955787 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-systemd-units\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955812 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-etc-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955842 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-ovn\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955859 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-etc-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955864 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-socket-dir-parent\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955890 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-ovn\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.955924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955936 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-socket-dir-parent\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955936 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-host\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955966 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57-host\") pod \"node-ca-ghl6s\" (UID: \"bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57\") " pod="openshift-image-registry/node-ca-ghl6s" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.955969 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-run\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956007 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-run\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956009 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-multus-certs\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956028 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb90a295-9822-4874-9014-0a91ac7a73c5-hosts-file\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956042 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-k8s-cni-cncf-io\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956056 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-bin\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956094 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-kubernetes\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956111 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-system-cni-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956095 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb90a295-9822-4874-9014-0a91ac7a73c5-hosts-file\") pod \"node-resolver-p6pc9\" (UID: \"bb90a295-9822-4874-9014-0a91ac7a73c5\") " pod="openshift-dns/node-resolver-p6pc9" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956129 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-conf\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956131 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-bin\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956137 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-multus-certs\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956109 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-run-k8s-cni-cncf-io\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956158 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-kubernetes\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956172 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cnibin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956189 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-system-cni-dir\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956209 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-modprobe-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956218 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-cnibin\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956239 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-sys-fs\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956260 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-var-lib-kubelet\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956292 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-system-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956302 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-sys-fs\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956311 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-modprobe-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956313 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-sys\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956340 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-conf\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956357 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-var-lib-kubelet\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956361 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ba49ca5-7b3e-42bc-aa61-59004aeb73dc-sys\") pod \"node-exporter-dcpkx\" (UID: \"0ba49ca5-7b3e-42bc-aa61-59004aeb73dc\") " pod="openshift-monitoring/node-exporter-dcpkx" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956362 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-log-socket\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956385 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-host\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956397 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-system-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956395 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-log-socket\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956401 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-socket-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956427 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-host\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956474 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-socket-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956499 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956526 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-run-openvswitch\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956534 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-kubelet\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956552 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-cnibin\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956567 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-kubelet-config\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956583 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-kubelet\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956612 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298e0d65-779b-4fd0-8931-43596a15d5fa-cnibin\") pod \"multus-additional-cni-plugins-gnmqz\" (UID: \"298e0d65-779b-4fd0-8931-43596a15d5fa\") " pod="openshift-multus/multus-additional-cni-plugins-gnmqz" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956621 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-kubelet\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956620 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3757b61-1b91-4739-a7fc-ecbfcb4feea6-kubelet-config\") pod \"global-pull-secret-syncer-jvsfx\" (UID: \"a3757b61-1b91-4739-a7fc-ecbfcb4feea6\") " pod="kube-system/global-pull-secret-syncer-jvsfx" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956640 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-host-var-lib-kubelet\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956643 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-netd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956671 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-sys\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956676 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-cni-netd\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956685 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956717 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-sys\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956736 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956734 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956773 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956798 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956812 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-sysctl-d\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956822 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-cni-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956823 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-systemd\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956840 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1e0c2b5-3116-4963-b1df-fef06ea58937-host-run-ovn-kubernetes\") pod \"ovnkube-node-x62cc\" (UID: \"e1e0c2b5-3116-4963-b1df-fef06ea58937\") " pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956844 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-registration-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956861 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-conf-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956863 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc3812fa-69b5-4e6e-bca9-91a3b191ed32-etc-systemd\") pod \"tuned-f2tdf\" (UID: \"dc3812fa-69b5-4e6e-bca9-91a3b191ed32\") " pod="openshift-cluster-node-tuning-operator/tuned-f2tdf" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956876 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-etc-kubernetes\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956881 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/018f82d1-73aa-4c50-bce5-6fef86b947b0-registration-dir\") pod \"aws-ebs-csi-driver-node-29tfd\" (UID: \"018f82d1-73aa-4c50-bce5-6fef86b947b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-29tfd" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956920 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-multus-conf-dir\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:04.957125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:04.956938 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f8d4383-c954-4fed-aadf-f1ee7c0eb24e-etc-kubernetes\") pod \"multus-lnxz5\" (UID: \"8f8d4383-c954-4fed-aadf-f1ee7c0eb24e\") " pod="openshift-multus/multus-lnxz5" Oct 30 19:48:05.052026 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:05.051991 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57b59b9797-ptgs8"] Oct 30 19:48:05.096349 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:05.096309 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:48:05.772339 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:05.772303 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:05.791654 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:05.791632 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:06.400947 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:06.400917 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:07.084456 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:07.084424 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cwb2s" Oct 30 19:48:07.497441 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:07.497407 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:07.501995 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:07.501974 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:08.284525 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:08.284491 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:08.500873 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:08.500839 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:10.198582 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:10.198557 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:11.184399 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:11.184356 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:11.565706 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:11.565614 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:11.607192 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:11.607168 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:48:11.711475 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:11.711430 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:12.225389 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:12.225362 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:12.225627 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:12.225615 6411 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:48:16.793097 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:16.793063 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:17.057070 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:17.056973 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:17.285245 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:17.285208 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:17.947141 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:17.947107 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:18.124688 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:18.124643 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:21.189211 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:21.189182 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:48:21.653323 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:21.653269 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:21.845515 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:21.845486 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:22.685453 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:22.685427 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5f9d8fd57c-dkv9w" Oct 30 19:48:24.647986 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:24.647952 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-44glf" Oct 30 19:48:27.361526 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:27.361491 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:27.646446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:27.646417 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:28.403460 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.403432 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x62cc" Oct 30 19:48:28.672943 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.672846 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 19:48:28.673291 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.673268 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="alertmanager" containerID="cri-o://1bd19a6f609bc25020a3c33285b08cf9d49013f0527c5dcdbf85bfa99f863959" gracePeriod=120 Oct 30 19:48:28.673375 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.673323 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy-metric" containerID="cri-o://2bc3842b2b8607e5e6a013304dc5d5c3908ce58fe0fb6a2406362be2c0b083d7" gracePeriod=120 Oct 30 19:48:28.673439 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.673375 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy-web" containerID="cri-o://5d66f1b4008772adaa5200c686c02bb67b02af6469da3cc7515f1e812e9b15a0" gracePeriod=120 Oct 30 19:48:28.673439 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.673388 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="prom-label-proxy" containerID="cri-o://f0063b501afc10b0774b76a383481b02f0dd79d21917143cb186dbb1b0ea2785" gracePeriod=120 Oct 30 19:48:28.673542 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.673410 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy" containerID="cri-o://84c9967ee25c8ae09bc22c3cc5c5ada991268f9dad75ff9579438c44d04e0e37" gracePeriod=120 Oct 30 19:48:28.673542 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.673456 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="config-reloader" containerID="cri-o://7eae5572e68c0c3cfeb00f38f811c3628fa3047d236e0a2a4dcebee06fe927af" gracePeriod=120 Oct 30 19:48:28.887613 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887581 6411 generic.go:356] "Generic (PLEG): container finished" podID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerID="f0063b501afc10b0774b76a383481b02f0dd79d21917143cb186dbb1b0ea2785" exitCode=0 Oct 30 19:48:28.887613 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887606 6411 generic.go:356] "Generic (PLEG): container finished" podID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerID="2bc3842b2b8607e5e6a013304dc5d5c3908ce58fe0fb6a2406362be2c0b083d7" exitCode=0 Oct 30 19:48:28.887613 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887613 6411 generic.go:356] "Generic (PLEG): container finished" podID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerID="84c9967ee25c8ae09bc22c3cc5c5ada991268f9dad75ff9579438c44d04e0e37" exitCode=0 Oct 30 19:48:28.887613 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887619 6411 generic.go:356] "Generic (PLEG): container finished" podID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerID="7eae5572e68c0c3cfeb00f38f811c3628fa3047d236e0a2a4dcebee06fe927af" exitCode=0 Oct 30 19:48:28.887821 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887625 6411 generic.go:356] "Generic (PLEG): container finished" podID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerID="1bd19a6f609bc25020a3c33285b08cf9d49013f0527c5dcdbf85bfa99f863959" exitCode=0 Oct 30 19:48:28.887821 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887655 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerDied","Data":"f0063b501afc10b0774b76a383481b02f0dd79d21917143cb186dbb1b0ea2785"} Oct 30 19:48:28.887821 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887716 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerDied","Data":"2bc3842b2b8607e5e6a013304dc5d5c3908ce58fe0fb6a2406362be2c0b083d7"} Oct 30 19:48:28.887821 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887742 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerDied","Data":"84c9967ee25c8ae09bc22c3cc5c5ada991268f9dad75ff9579438c44d04e0e37"} Oct 30 19:48:28.887821 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887767 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerDied","Data":"7eae5572e68c0c3cfeb00f38f811c3628fa3047d236e0a2a4dcebee06fe927af"} Oct 30 19:48:28.887821 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:28.887790 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerDied","Data":"1bd19a6f609bc25020a3c33285b08cf9d49013f0527c5dcdbf85bfa99f863959"} Oct 30 19:48:29.894629 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.894594 6411 generic.go:356] "Generic (PLEG): container finished" podID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerID="5d66f1b4008772adaa5200c686c02bb67b02af6469da3cc7515f1e812e9b15a0" exitCode=0 Oct 30 19:48:29.894987 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.894665 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerDied","Data":"5d66f1b4008772adaa5200c686c02bb67b02af6469da3cc7515f1e812e9b15a0"} Oct 30 19:48:29.928509 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.928484 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:29.959324 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959289 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959508 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959358 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959508 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959392 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959508 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959454 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959508 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959484 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959510 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959541 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959573 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959605 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959642 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959679 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.959735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959718 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.960079 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959747 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume\") pod \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\" (UID: \"55f5f63c-a55a-4f7c-858c-bb3051ef73be\") " Oct 30 19:48:29.960079 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.959807 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:29.960079 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.960055 6411 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-main-db\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:29.960828 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.960592 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:29.961380 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.961119 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:29.963143 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.963114 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "kube-api-access-pmc7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:48:29.963571 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.963550 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:29.963791 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.963705 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:29.963868 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.963834 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:29.964015 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.963991 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:29.964140 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.964109 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:29.964265 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.964242 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:29.965276 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.965257 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:48:29.967359 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.967242 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:29.973885 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:29.973830 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config" pod "55f5f63c-a55a-4f7c-858c-bb3051ef73be" (UID: "55f5f63c-a55a-4f7c-858c-bb3051ef73be"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:30.061017 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.060988 6411 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-web-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061022 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmc7x\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-kube-api-access-pmc7x\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061037 6411 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55f5f63c-a55a-4f7c-858c-bb3051ef73be-tls-assets\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061046 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-metrics-client-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061055 6411 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-main-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061066 6411 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061076 6411 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061085 6411 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55f5f63c-a55a-4f7c-858c-bb3051ef73be-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061095 6411 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-volume\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061122 6411 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55f5f63c-a55a-4f7c-858c-bb3051ef73be-config-out\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061131 6411 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.061175 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.061141 6411 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55f5f63c-a55a-4f7c-858c-bb3051ef73be-cluster-tls-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:30.816258 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.816222 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-57b59b9797-ptgs8" podUID="42a9532b-3f87-4fac-816a-d293b921ce5c" containerName="console" containerID="cri-o://59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913" gracePeriod=15 Oct 30 19:48:30.901403 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.901366 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55f5f63c-a55a-4f7c-858c-bb3051ef73be","Type":"ContainerDied","Data":"d03e243eafcb3ce5236c92e0eb4fe70065a94c06f91e1cba3ea5b8be5a092566"} Oct 30 19:48:30.901856 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.901426 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 19:48:30.901856 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.901465 6411 scope.go:122] "RemoveContainer" containerID="f0063b501afc10b0774b76a383481b02f0dd79d21917143cb186dbb1b0ea2785" Oct 30 19:48:30.926816 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.926794 6411 scope.go:122] "RemoveContainer" containerID="2bc3842b2b8607e5e6a013304dc5d5c3908ce58fe0fb6a2406362be2c0b083d7" Oct 30 19:48:30.934191 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.934158 6411 scope.go:122] "RemoveContainer" containerID="84c9967ee25c8ae09bc22c3cc5c5ada991268f9dad75ff9579438c44d04e0e37" Oct 30 19:48:30.942353 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.942333 6411 scope.go:122] "RemoveContainer" containerID="5d66f1b4008772adaa5200c686c02bb67b02af6469da3cc7515f1e812e9b15a0" Oct 30 19:48:30.949766 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.949743 6411 scope.go:122] "RemoveContainer" containerID="7eae5572e68c0c3cfeb00f38f811c3628fa3047d236e0a2a4dcebee06fe927af" Oct 30 19:48:30.957135 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.957114 6411 scope.go:122] "RemoveContainer" containerID="1bd19a6f609bc25020a3c33285b08cf9d49013f0527c5dcdbf85bfa99f863959" Oct 30 19:48:30.965273 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.965249 6411 scope.go:122] "RemoveContainer" containerID="9503fd4b4d23f77269e7bdb0f71b093fe36d61208fc9c169139ba6cd9d316a2c" Oct 30 19:48:30.973660 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.973612 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 19:48:30.986485 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:30.986454 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 19:48:31.077288 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.077250 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:31.171880 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.171844 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert\") pod \"42a9532b-3f87-4fac-816a-d293b921ce5c\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " Oct 30 19:48:31.172054 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.171918 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle\") pod \"42a9532b-3f87-4fac-816a-d293b921ce5c\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " Oct 30 19:48:31.172054 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.171962 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z\" (UniqueName: \"kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z\") pod \"42a9532b-3f87-4fac-816a-d293b921ce5c\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " Oct 30 19:48:31.172054 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.171998 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert\") pod \"42a9532b-3f87-4fac-816a-d293b921ce5c\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " Oct 30 19:48:31.172054 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172038 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config\") pod \"42a9532b-3f87-4fac-816a-d293b921ce5c\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " Oct 30 19:48:31.172277 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172070 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca\") pod \"42a9532b-3f87-4fac-816a-d293b921ce5c\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " Oct 30 19:48:31.172277 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172117 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config\") pod \"42a9532b-3f87-4fac-816a-d293b921ce5c\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " Oct 30 19:48:31.172277 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172155 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp\" (UniqueName: \"kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp\") pod \"42a9532b-3f87-4fac-816a-d293b921ce5c\" (UID: \"42a9532b-3f87-4fac-816a-d293b921ce5c\") " Oct 30 19:48:31.172452 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172341 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle" pod "42a9532b-3f87-4fac-816a-d293b921ce5c" (UID: "42a9532b-3f87-4fac-816a-d293b921ce5c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:31.172560 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172533 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp" pod "42a9532b-3f87-4fac-816a-d293b921ce5c" (UID: "42a9532b-3f87-4fac-816a-d293b921ce5c"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:31.172720 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172670 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca" pod "42a9532b-3f87-4fac-816a-d293b921ce5c" (UID: "42a9532b-3f87-4fac-816a-d293b921ce5c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:31.172720 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172691 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert" pod "42a9532b-3f87-4fac-816a-d293b921ce5c" (UID: "42a9532b-3f87-4fac-816a-d293b921ce5c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:31.172859 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.172845 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config" pod "42a9532b-3f87-4fac-816a-d293b921ce5c" (UID: "42a9532b-3f87-4fac-816a-d293b921ce5c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:31.174482 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.174459 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config" pod "42a9532b-3f87-4fac-816a-d293b921ce5c" (UID: "42a9532b-3f87-4fac-816a-d293b921ce5c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:31.174769 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.174743 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert" pod "42a9532b-3f87-4fac-816a-d293b921ce5c" (UID: "42a9532b-3f87-4fac-816a-d293b921ce5c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:31.174847 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.174796 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z" pod "42a9532b-3f87-4fac-816a-d293b921ce5c" (UID: "42a9532b-3f87-4fac-816a-d293b921ce5c"). InnerVolumeSpecName "kube-api-access-4z65z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:48:31.273443 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.273359 6411 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-serving-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:31.273443 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.273390 6411 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-trusted-ca-bundle\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:31.273443 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.273401 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z65z\" (UniqueName: \"kubernetes.io/projected/42a9532b-3f87-4fac-816a-d293b921ce5c-kube-api-access-4z65z\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:31.273443 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.273410 6411 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-oauth-serving-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:31.273443 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.273419 6411 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-console-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:31.273443 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.273429 6411 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42a9532b-3f87-4fac-816a-d293b921ce5c-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:31.273443 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.273437 6411 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42a9532b-3f87-4fac-816a-d293b921ce5c-console-oauth-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:31.273443 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.273445 6411 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42a9532b-3f87-4fac-816a-d293b921ce5c-tmp\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:31.609959 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.609863 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bvdgb" Oct 30 19:48:31.730758 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.730726 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" path="/var/lib/kubelet/pods/55f5f63c-a55a-4f7c-858c-bb3051ef73be/volumes" Oct 30 19:48:31.906113 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.906076 6411 generic.go:356] "Generic (PLEG): container finished" podID="42a9532b-3f87-4fac-816a-d293b921ce5c" containerID="59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913" exitCode=0 Oct 30 19:48:31.906553 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.906122 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b59b9797-ptgs8" Oct 30 19:48:31.906553 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.906167 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b59b9797-ptgs8" event={"ID":"42a9532b-3f87-4fac-816a-d293b921ce5c","Type":"ContainerDied","Data":"59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913"} Oct 30 19:48:31.906553 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.906211 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b59b9797-ptgs8" event={"ID":"42a9532b-3f87-4fac-816a-d293b921ce5c","Type":"ContainerDied","Data":"38efa6c6146cff0df453fce2e0de42b581de3b4cb80b73e6c2279da08ddeec2d"} Oct 30 19:48:31.906553 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.906232 6411 scope.go:122] "RemoveContainer" containerID="59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913" Oct 30 19:48:31.913941 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.913923 6411 scope.go:122] "RemoveContainer" containerID="59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913" Oct 30 19:48:31.914217 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:31.914199 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913\": container with ID starting with 59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913 not found: ID does not exist" containerID="59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913" Oct 30 19:48:31.914260 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.914228 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913"} err="failed to get container status \"59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913\": rpc error: code = NotFound desc = could not find container \"59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913\": container with ID starting with 59138b88839612fab359895ca36a80b6c79079e3bc33a1e2b956a2f109f28913 not found: ID does not exist" Oct 30 19:48:31.966085 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.966050 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57b59b9797-ptgs8"] Oct 30 19:48:31.978372 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:31.978344 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57b59b9797-ptgs8"] Oct 30 19:48:33.346786 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.345372 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:48:33.346786 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.345749 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="prometheus" containerID="cri-o://709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" gracePeriod=600 Oct 30 19:48:33.346786 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.345761 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" gracePeriod=600 Oct 30 19:48:33.346786 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.345938 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy" containerID="cri-o://3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" gracePeriod=600 Oct 30 19:48:33.346786 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.345964 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="thanos-sidecar" containerID="cri-o://da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" gracePeriod=600 Oct 30 19:48:33.346786 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.346027 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="config-reloader" containerID="cri-o://339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" gracePeriod=600 Oct 30 19:48:33.346786 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.346040 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy-web" containerID="cri-o://6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" gracePeriod=600 Oct 30 19:48:33.618630 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.618601 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:33.695628 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695599 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.695794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695639 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.695794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695666 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.695794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695695 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.695794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695715 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.695794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695740 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.695794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695776 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695820 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695842 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695869 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695917 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695947 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.695978 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696008 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696067 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696097 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696116 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696125 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696183 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696202 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696241 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696270 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy\") pod \"8a61eb1f-e59d-4050-b108-20b6408b819a\" (UID: \"8a61eb1f-e59d-4050-b108-20b6408b819a\") " Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696493 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696498 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696660 6411 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-metrics-client-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696680 6411 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.696692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696696 6411 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.697371 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.696710 6411 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.698626 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.698571 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:33.699211 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.698886 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:33.700304 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.700204 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:33.701076 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.700932 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "kube-api-access-wzrnv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:48:33.702384 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.702355 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.702683 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.702653 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:33.703626 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.703600 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.704004 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.703980 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.704314 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.704273 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.704414 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.704401 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:48:33.704516 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.704482 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.704566 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.704525 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:33.704633 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.704605 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.704676 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.704657 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.707113 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.706019 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.714593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.714564 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config" pod "8a61eb1f-e59d-4050-b108-20b6408b819a" (UID: "8a61eb1f-e59d-4050-b108-20b6408b819a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:48:33.731378 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.731347 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42a9532b-3f87-4fac-816a-d293b921ce5c" path="/var/lib/kubelet/pods/42a9532b-3f87-4fac-816a-d293b921ce5c/volumes" Oct 30 19:48:33.798081 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798048 6411 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-config-out\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798081 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798078 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzrnv\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-kube-api-access-wzrnv\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798081 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798089 6411 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798100 6411 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798109 6411 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798118 6411 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798128 6411 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798136 6411 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-db\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798144 6411 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-grpc-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798152 6411 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-metrics-client-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798160 6411 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798169 6411 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798177 6411 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a61eb1f-e59d-4050-b108-20b6408b819a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798185 6411 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a61eb1f-e59d-4050-b108-20b6408b819a-tls-assets\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798213 6411 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-web-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.798305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.798221 6411 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a61eb1f-e59d-4050-b108-20b6408b819a-secret-kube-rbac-proxy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:48:33.919015 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.918984 6411 generic.go:356] "Generic (PLEG): container finished" podID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" exitCode=0 Oct 30 19:48:33.919015 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919008 6411 generic.go:356] "Generic (PLEG): container finished" podID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" exitCode=0 Oct 30 19:48:33.919015 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919014 6411 generic.go:356] "Generic (PLEG): container finished" podID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" exitCode=0 Oct 30 19:48:33.919015 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919021 6411 generic.go:356] "Generic (PLEG): container finished" podID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" exitCode=0 Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919028 6411 generic.go:356] "Generic (PLEG): container finished" podID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" exitCode=0 Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919033 6411 generic.go:356] "Generic (PLEG): container finished" podID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" exitCode=0 Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919081 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919076 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerDied","Data":"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5"} Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919218 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerDied","Data":"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2"} Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919247 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerDied","Data":"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449"} Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919271 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerDied","Data":"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795"} Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919284 6411 scope.go:122] "RemoveContainer" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919297 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerDied","Data":"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430"} Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919320 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerDied","Data":"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49"} Oct 30 19:48:33.919373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.919343 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a61eb1f-e59d-4050-b108-20b6408b819a","Type":"ContainerDied","Data":"b932d03caaef8b6a30aba82ea26989b0a0e4f4c7849e38bfe5671eb540886c72"} Oct 30 19:48:33.927100 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.927085 6411 scope.go:122] "RemoveContainer" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" Oct 30 19:48:33.937296 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.934862 6411 scope.go:122] "RemoveContainer" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" Oct 30 19:48:33.943370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.943353 6411 scope.go:122] "RemoveContainer" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" Oct 30 19:48:33.949526 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.949511 6411 scope.go:122] "RemoveContainer" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" Oct 30 19:48:33.955984 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.955968 6411 scope.go:122] "RemoveContainer" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" Oct 30 19:48:33.962555 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.962540 6411 scope.go:122] "RemoveContainer" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" Oct 30 19:48:33.968543 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.968529 6411 scope.go:122] "RemoveContainer" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" Oct 30 19:48:33.968783 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:33.968768 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": container with ID starting with 3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5 not found: ID does not exist" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" Oct 30 19:48:33.968828 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.968790 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5"} err="failed to get container status \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": rpc error: code = NotFound desc = could not find container \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": container with ID starting with 3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5 not found: ID does not exist" Oct 30 19:48:33.968828 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.968819 6411 scope.go:122] "RemoveContainer" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" Oct 30 19:48:33.969057 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:33.969038 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": container with ID starting with 3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2 not found: ID does not exist" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" Oct 30 19:48:33.969104 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.969062 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2"} err="failed to get container status \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": rpc error: code = NotFound desc = could not find container \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": container with ID starting with 3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2 not found: ID does not exist" Oct 30 19:48:33.969104 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.969078 6411 scope.go:122] "RemoveContainer" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" Oct 30 19:48:33.969301 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:33.969285 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": container with ID starting with 6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449 not found: ID does not exist" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" Oct 30 19:48:33.969341 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.969307 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449"} err="failed to get container status \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": rpc error: code = NotFound desc = could not find container \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": container with ID starting with 6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449 not found: ID does not exist" Oct 30 19:48:33.969341 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.969320 6411 scope.go:122] "RemoveContainer" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" Oct 30 19:48:33.969538 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:33.969524 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": container with ID starting with da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795 not found: ID does not exist" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" Oct 30 19:48:33.969580 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.969542 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795"} err="failed to get container status \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": rpc error: code = NotFound desc = could not find container \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": container with ID starting with da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795 not found: ID does not exist" Oct 30 19:48:33.969580 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.969555 6411 scope.go:122] "RemoveContainer" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" Oct 30 19:48:33.969751 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:33.969736 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": container with ID starting with 339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430 not found: ID does not exist" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" Oct 30 19:48:33.969790 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.969754 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430"} err="failed to get container status \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": rpc error: code = NotFound desc = could not find container \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": container with ID starting with 339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430 not found: ID does not exist" Oct 30 19:48:33.969790 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.969765 6411 scope.go:122] "RemoveContainer" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" Oct 30 19:48:33.970083 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:33.970067 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": container with ID starting with 709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49 not found: ID does not exist" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" Oct 30 19:48:33.970130 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.970086 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49"} err="failed to get container status \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": rpc error: code = NotFound desc = could not find container \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": container with ID starting with 709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49 not found: ID does not exist" Oct 30 19:48:33.970130 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.970096 6411 scope.go:122] "RemoveContainer" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" Oct 30 19:48:33.970345 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:33.970331 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": container with ID starting with 3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7 not found: ID does not exist" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" Oct 30 19:48:33.970392 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.970348 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7"} err="failed to get container status \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": rpc error: code = NotFound desc = could not find container \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": container with ID starting with 3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7 not found: ID does not exist" Oct 30 19:48:33.970392 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.970358 6411 scope.go:122] "RemoveContainer" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" Oct 30 19:48:33.970582 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.970564 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5"} err="failed to get container status \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": rpc error: code = NotFound desc = could not find container \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": container with ID starting with 3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5 not found: ID does not exist" Oct 30 19:48:33.970626 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.970591 6411 scope.go:122] "RemoveContainer" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" Oct 30 19:48:33.970809 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.970791 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2"} err="failed to get container status \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": rpc error: code = NotFound desc = could not find container \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": container with ID starting with 3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2 not found: ID does not exist" Oct 30 19:48:33.970879 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.970813 6411 scope.go:122] "RemoveContainer" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" Oct 30 19:48:33.971070 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971051 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449"} err="failed to get container status \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": rpc error: code = NotFound desc = could not find container \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": container with ID starting with 6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449 not found: ID does not exist" Oct 30 19:48:33.971119 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971070 6411 scope.go:122] "RemoveContainer" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" Oct 30 19:48:33.971265 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971244 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795"} err="failed to get container status \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": rpc error: code = NotFound desc = could not find container \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": container with ID starting with da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795 not found: ID does not exist" Oct 30 19:48:33.971305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971266 6411 scope.go:122] "RemoveContainer" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" Oct 30 19:48:33.971481 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971465 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430"} err="failed to get container status \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": rpc error: code = NotFound desc = could not find container \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": container with ID starting with 339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430 not found: ID does not exist" Oct 30 19:48:33.971529 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971481 6411 scope.go:122] "RemoveContainer" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" Oct 30 19:48:33.971686 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971667 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49"} err="failed to get container status \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": rpc error: code = NotFound desc = could not find container \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": container with ID starting with 709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49 not found: ID does not exist" Oct 30 19:48:33.971756 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971687 6411 scope.go:122] "RemoveContainer" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" Oct 30 19:48:33.971959 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971943 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7"} err="failed to get container status \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": rpc error: code = NotFound desc = could not find container \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": container with ID starting with 3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7 not found: ID does not exist" Oct 30 19:48:33.972009 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.971960 6411 scope.go:122] "RemoveContainer" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" Oct 30 19:48:33.972170 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.972152 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5"} err="failed to get container status \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": rpc error: code = NotFound desc = could not find container \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": container with ID starting with 3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5 not found: ID does not exist" Oct 30 19:48:33.972239 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.972171 6411 scope.go:122] "RemoveContainer" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" Oct 30 19:48:33.972432 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.972402 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2"} err="failed to get container status \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": rpc error: code = NotFound desc = could not find container \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": container with ID starting with 3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2 not found: ID does not exist" Oct 30 19:48:33.972474 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.972432 6411 scope.go:122] "RemoveContainer" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" Oct 30 19:48:33.972646 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.972629 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449"} err="failed to get container status \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": rpc error: code = NotFound desc = could not find container \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": container with ID starting with 6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449 not found: ID does not exist" Oct 30 19:48:33.972714 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.972647 6411 scope.go:122] "RemoveContainer" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" Oct 30 19:48:33.972868 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.972850 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795"} err="failed to get container status \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": rpc error: code = NotFound desc = could not find container \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": container with ID starting with da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795 not found: ID does not exist" Oct 30 19:48:33.972959 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.972869 6411 scope.go:122] "RemoveContainer" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" Oct 30 19:48:33.973098 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973081 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430"} err="failed to get container status \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": rpc error: code = NotFound desc = could not find container \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": container with ID starting with 339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430 not found: ID does not exist" Oct 30 19:48:33.973163 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973099 6411 scope.go:122] "RemoveContainer" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" Oct 30 19:48:33.973304 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973288 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49"} err="failed to get container status \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": rpc error: code = NotFound desc = could not find container \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": container with ID starting with 709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49 not found: ID does not exist" Oct 30 19:48:33.973369 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973305 6411 scope.go:122] "RemoveContainer" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" Oct 30 19:48:33.973499 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973484 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7"} err="failed to get container status \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": rpc error: code = NotFound desc = could not find container \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": container with ID starting with 3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7 not found: ID does not exist" Oct 30 19:48:33.973565 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973500 6411 scope.go:122] "RemoveContainer" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" Oct 30 19:48:33.973713 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973697 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5"} err="failed to get container status \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": rpc error: code = NotFound desc = could not find container \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": container with ID starting with 3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5 not found: ID does not exist" Oct 30 19:48:33.973779 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973714 6411 scope.go:122] "RemoveContainer" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" Oct 30 19:48:33.973949 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973928 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2"} err="failed to get container status \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": rpc error: code = NotFound desc = could not find container \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": container with ID starting with 3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2 not found: ID does not exist" Oct 30 19:48:33.974025 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.973950 6411 scope.go:122] "RemoveContainer" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" Oct 30 19:48:33.974167 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.974151 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449"} err="failed to get container status \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": rpc error: code = NotFound desc = could not find container \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": container with ID starting with 6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449 not found: ID does not exist" Oct 30 19:48:33.974232 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.974168 6411 scope.go:122] "RemoveContainer" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" Oct 30 19:48:33.974371 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.974355 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795"} err="failed to get container status \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": rpc error: code = NotFound desc = could not find container \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": container with ID starting with da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795 not found: ID does not exist" Oct 30 19:48:33.974436 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.974372 6411 scope.go:122] "RemoveContainer" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" Oct 30 19:48:33.974582 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.974565 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430"} err="failed to get container status \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": rpc error: code = NotFound desc = could not find container \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": container with ID starting with 339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430 not found: ID does not exist" Oct 30 19:48:33.974647 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.974582 6411 scope.go:122] "RemoveContainer" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" Oct 30 19:48:33.974805 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.974783 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49"} err="failed to get container status \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": rpc error: code = NotFound desc = could not find container \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": container with ID starting with 709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49 not found: ID does not exist" Oct 30 19:48:33.974842 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.974814 6411 scope.go:122] "RemoveContainer" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" Oct 30 19:48:33.975055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975029 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7"} err="failed to get container status \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": rpc error: code = NotFound desc = could not find container \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": container with ID starting with 3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7 not found: ID does not exist" Oct 30 19:48:33.975055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975048 6411 scope.go:122] "RemoveContainer" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" Oct 30 19:48:33.975255 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975239 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5"} err="failed to get container status \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": rpc error: code = NotFound desc = could not find container \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": container with ID starting with 3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5 not found: ID does not exist" Oct 30 19:48:33.975303 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975256 6411 scope.go:122] "RemoveContainer" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" Oct 30 19:48:33.975503 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975484 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2"} err="failed to get container status \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": rpc error: code = NotFound desc = could not find container \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": container with ID starting with 3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2 not found: ID does not exist" Oct 30 19:48:33.975572 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975505 6411 scope.go:122] "RemoveContainer" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" Oct 30 19:48:33.975739 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975717 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449"} err="failed to get container status \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": rpc error: code = NotFound desc = could not find container \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": container with ID starting with 6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449 not found: ID does not exist" Oct 30 19:48:33.975791 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975741 6411 scope.go:122] "RemoveContainer" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" Oct 30 19:48:33.975974 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975955 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795"} err="failed to get container status \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": rpc error: code = NotFound desc = could not find container \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": container with ID starting with da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795 not found: ID does not exist" Oct 30 19:48:33.976062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.975975 6411 scope.go:122] "RemoveContainer" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" Oct 30 19:48:33.976220 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.976201 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430"} err="failed to get container status \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": rpc error: code = NotFound desc = could not find container \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": container with ID starting with 339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430 not found: ID does not exist" Oct 30 19:48:33.976260 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.976228 6411 scope.go:122] "RemoveContainer" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" Oct 30 19:48:33.976459 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.976442 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49"} err="failed to get container status \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": rpc error: code = NotFound desc = could not find container \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": container with ID starting with 709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49 not found: ID does not exist" Oct 30 19:48:33.976527 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.976460 6411 scope.go:122] "RemoveContainer" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" Oct 30 19:48:33.976713 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.976694 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7"} err="failed to get container status \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": rpc error: code = NotFound desc = could not find container \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": container with ID starting with 3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7 not found: ID does not exist" Oct 30 19:48:33.976759 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.976714 6411 scope.go:122] "RemoveContainer" containerID="3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5" Oct 30 19:48:33.976957 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.976939 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5"} err="failed to get container status \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": rpc error: code = NotFound desc = could not find container \"3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5\": container with ID starting with 3daac73ea20b04da1fbc8e822a1928168387c1eb34cb9cde6696879ba6da66b5 not found: ID does not exist" Oct 30 19:48:33.977003 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.976957 6411 scope.go:122] "RemoveContainer" containerID="3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2" Oct 30 19:48:33.977183 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.977163 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2"} err="failed to get container status \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": rpc error: code = NotFound desc = could not find container \"3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2\": container with ID starting with 3c5da668fa69081509a328702552e7aea8b1ce89c1f19d1b7878b390053efbb2 not found: ID does not exist" Oct 30 19:48:33.977183 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.977184 6411 scope.go:122] "RemoveContainer" containerID="6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449" Oct 30 19:48:33.977430 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.977412 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449"} err="failed to get container status \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": rpc error: code = NotFound desc = could not find container \"6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449\": container with ID starting with 6fa2c2ce8834da228bd6b0d6081143c0b80edd9cc3cc3b7d8918451a76c9a449 not found: ID does not exist" Oct 30 19:48:33.977430 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.977430 6411 scope.go:122] "RemoveContainer" containerID="da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795" Oct 30 19:48:33.977675 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.977657 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795"} err="failed to get container status \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": rpc error: code = NotFound desc = could not find container \"da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795\": container with ID starting with da5fc02bd65e8a79ce5379dd7085767087141dedc1d4fb466942ecfd65c64795 not found: ID does not exist" Oct 30 19:48:33.977751 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.977675 6411 scope.go:122] "RemoveContainer" containerID="339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430" Oct 30 19:48:33.978019 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.977949 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430"} err="failed to get container status \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": rpc error: code = NotFound desc = could not find container \"339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430\": container with ID starting with 339a567e20b5307cddbd8901ccb2d2843a443c248801b5ce5bc463b518cc2430 not found: ID does not exist" Oct 30 19:48:33.978019 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.977971 6411 scope.go:122] "RemoveContainer" containerID="709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49" Oct 30 19:48:33.978266 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.978222 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49"} err="failed to get container status \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": rpc error: code = NotFound desc = could not find container \"709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49\": container with ID starting with 709c8e8390745fdcda75431b5dc835b1f306f56172711acb9818f32ea8cd1e49 not found: ID does not exist" Oct 30 19:48:33.978266 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.978250 6411 scope.go:122] "RemoveContainer" containerID="3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7" Oct 30 19:48:33.978496 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.978474 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7"} err="failed to get container status \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": rpc error: code = NotFound desc = could not find container \"3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7\": container with ID starting with 3d9e6a1d242a48dc9c65fb75b67696d19528f75ae7ff005e0f6635baee253ea7 not found: ID does not exist" Oct 30 19:48:33.979174 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.979154 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:48:33.985269 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:33.985253 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:48:34.035416 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035386 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:48:34.035698 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035683 6411 cpu_manager.go:414] "containerMap: removing container" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="init-config-reloader" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035702 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="init-config-reloader" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035727 6411 cpu_manager.go:414] "containerMap: removing container" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="alertmanager" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035735 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="alertmanager" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035748 6411 cpu_manager.go:414] "containerMap: removing container" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="thanos-sidecar" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035756 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="thanos-sidecar" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035766 6411 cpu_manager.go:414] "containerMap: removing container" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy-metric" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035773 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy-metric" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035785 6411 cpu_manager.go:414] "containerMap: removing container" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy" Oct 30 19:48:34.035798 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035792 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035805 6411 cpu_manager.go:414] "containerMap: removing container" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy-web" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035812 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy-web" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035824 6411 cpu_manager.go:414] "containerMap: removing container" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="prom-label-proxy" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035832 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="prom-label-proxy" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035850 6411 cpu_manager.go:414] "containerMap: removing container" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy-thanos" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035858 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy-thanos" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035867 6411 cpu_manager.go:414] "containerMap: removing container" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy-web" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035874 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy-web" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035884 6411 cpu_manager.go:414] "containerMap: removing container" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035891 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035926 6411 cpu_manager.go:414] "containerMap: removing container" podUID="42a9532b-3f87-4fac-816a-d293b921ce5c" containerName="console" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035933 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42a9532b-3f87-4fac-816a-d293b921ce5c" containerName="console" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035944 6411 cpu_manager.go:414] "containerMap: removing container" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="prometheus" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035951 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="prometheus" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035960 6411 cpu_manager.go:414] "containerMap: removing container" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035967 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035980 6411 cpu_manager.go:414] "containerMap: removing container" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035986 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.035998 6411 cpu_manager.go:414] "containerMap: removing container" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="init-config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036005 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="init-config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036073 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="prom-label-proxy" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036088 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy-thanos" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036097 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy-web" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036105 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036113 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="42a9532b-3f87-4fac-816a-d293b921ce5c" containerName="console" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036120 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="prometheus" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036127 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036136 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036145 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="init-config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036153 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="init-config-reloader" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036163 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="alertmanager" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036171 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="thanos-sidecar" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036181 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="55f5f63c-a55a-4f7c-858c-bb3051ef73be" containerName="kube-rbac-proxy-metric" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036191 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy" Oct 30 19:48:34.036257 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.036198 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" containerName="kube-rbac-proxy-web" Oct 30 19:48:34.040213 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.040191 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.059581 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.059552 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 19:48:34.059717 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.059613 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 19:48:34.060081 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060049 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 19:48:34.060081 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060068 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 19:48:34.060250 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060088 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 19:48:34.060250 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060107 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 19:48:34.060250 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060053 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 19:48:34.060250 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060153 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 19:48:34.060250 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060160 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9uipjplehrdud\"" Oct 30 19:48:34.060250 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060180 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 19:48:34.060250 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.060214 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 19:48:34.072355 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.072325 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k6b6j\"" Oct 30 19:48:34.072834 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.072811 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 19:48:34.073070 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.072886 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 19:48:34.073205 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.073188 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 19:48:34.073539 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.073520 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 19:48:34.073991 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.073973 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:48:34.100533 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100505 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100533 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100535 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100754 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100564 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100754 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100672 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100754 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100700 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100754 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100718 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100754 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100737 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-config-out\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100950 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100812 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100950 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100846 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100950 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100861 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100950 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100875 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.100950 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100890 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.101150 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.100990 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.101150 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.101033 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-web-config\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.101150 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.101055 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.101150 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.101084 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.101150 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.101128 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-config\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.101150 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.101150 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.101343 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.101192 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.101343 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.101212 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wcjv\" (UniqueName: \"kubernetes.io/projected/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-kube-api-access-4wcjv\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202382 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202295 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202382 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202335 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202382 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202352 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-web-config\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202617 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202476 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202617 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202521 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202617 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202560 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-config\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202617 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202599 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202814 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202629 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202814 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202654 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wcjv\" (UniqueName: \"kubernetes.io/projected/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-kube-api-access-4wcjv\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202814 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202689 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202814 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202714 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202814 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202751 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.202814 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202791 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202820 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202829 6411 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202844 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202868 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-config-out\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202939 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202973 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.202999 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.203020 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203055 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.203019 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203506 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.203306 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203506 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.203347 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203506 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.203357 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.203816 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.203736 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.205659 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.205629 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.206383 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.206104 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-config\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.206383 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.206296 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.206383 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.206319 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.206632 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.206420 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.206632 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.206305 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.206632 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.206494 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-web-config\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.206996 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.206974 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.207090 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.206982 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.208130 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.208110 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-config-out\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.208190 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.208120 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.208610 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.208594 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.209138 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.209123 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.209727 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.209709 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.228349 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.228327 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wcjv\" (UniqueName: \"kubernetes.io/projected/a3bbea30-36c9-4cf5-b9c6-782a0f9c6662-kube-api-access-4wcjv\") pod \"prometheus-k8s-0\" (UID: \"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.361834 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.361796 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:34.693345 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.693318 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 19:48:34.696316 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:34.696289 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3bbea30_36c9_4cf5_b9c6_782a0f9c6662.slice/crio-4a92c66866e80c09b2d63d3ec2fe0e6c8fe59ea43373cdb2bacddc1ca35651e4 WatchSource:0}: Error finding container 4a92c66866e80c09b2d63d3ec2fe0e6c8fe59ea43373cdb2bacddc1ca35651e4: Status 404 returned error can't find the container with id 4a92c66866e80c09b2d63d3ec2fe0e6c8fe59ea43373cdb2bacddc1ca35651e4 Oct 30 19:48:34.924045 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.924013 6411 generic.go:356] "Generic (PLEG): container finished" podID="a3bbea30-36c9-4cf5-b9c6-782a0f9c6662" containerID="f583187fcd45588303fb1981273be22f3aff112b43b82a31c5391051943eb411" exitCode=0 Oct 30 19:48:34.924195 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.924100 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662","Type":"ContainerDied","Data":"f583187fcd45588303fb1981273be22f3aff112b43b82a31c5391051943eb411"} Oct 30 19:48:34.924195 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:34.924162 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662","Type":"ContainerStarted","Data":"4a92c66866e80c09b2d63d3ec2fe0e6c8fe59ea43373cdb2bacddc1ca35651e4"} Oct 30 19:48:35.100821 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:48:35.100792 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:48:35.730651 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:35.730611 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a61eb1f-e59d-4050-b108-20b6408b819a" path="/var/lib/kubelet/pods/8a61eb1f-e59d-4050-b108-20b6408b819a/volumes" Oct 30 19:48:35.931611 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:35.931574 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662","Type":"ContainerStarted","Data":"bf14e4fd435fbf59465e589281cce332343f3575896f3f92cd8da6642dd51a10"} Oct 30 19:48:35.931774 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:35.931655 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662","Type":"ContainerStarted","Data":"c8acaeaced8dcf986205055e96cc716ce8e31a82ff9e3adf330172f9334a6bbe"} Oct 30 19:48:35.931774 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:35.931695 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662","Type":"ContainerStarted","Data":"e1f4094ccce5625aee2ff03e030f41f4d4506cf59390648a6a36821b491c9087"} Oct 30 19:48:35.931774 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:35.931732 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662","Type":"ContainerStarted","Data":"038aa17beae836aed924ad9b077946b5495af30077eee3e9d949dbf4350ffb22"} Oct 30 19:48:35.931774 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:35.931767 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662","Type":"ContainerStarted","Data":"9ff45813d11e940671e2b3443c309f9490c9efd9d65b6a5f692c2af42136918b"} Oct 30 19:48:35.931892 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:35.931799 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a3bbea30-36c9-4cf5-b9c6-782a0f9c6662","Type":"ContainerStarted","Data":"564f5d5197f525a7089669f8628e5663ac23c2d38c9936b1fa8cc8572ba1f514"} Oct 30 19:48:36.019604 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.019492 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.000004379 podStartE2EDuration="3.019475208s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.470829ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:36.017911114 +0000 UTC m=+32.418083112" watchObservedRunningTime="2026-10-30 19:48:36.019475208 +0000 UTC m=+32.419647223" Oct 30 19:48:36.063155 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.063129 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7986965f75-j6r64"] Oct 30 19:48:36.067127 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.067112 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.100466 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.100422 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7986965f75-j6r64"] Oct 30 19:48:36.119854 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.119834 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-config\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.119969 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.119869 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gpgm\" (UniqueName: \"kubernetes.io/projected/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-kube-api-access-5gpgm\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.119969 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.119911 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-trusted-ca-bundle\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.119969 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.119963 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-oauth-config\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.120101 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.119985 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-tmp\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.120101 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.120007 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-service-ca\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.120101 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.120025 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-serving-cert\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.120208 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.120105 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-oauth-serving-cert\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221316 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221289 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-trusted-ca-bundle\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221447 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221323 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-oauth-config\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221447 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221338 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-tmp\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221447 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221363 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-service-ca\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221447 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221387 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-serving-cert\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221447 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221427 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-oauth-serving-cert\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221684 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221451 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-config\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221684 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221487 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gpgm\" (UniqueName: \"kubernetes.io/projected/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-kube-api-access-5gpgm\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.221789 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.221749 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-tmp\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.222242 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.222221 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-config\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.222336 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.222254 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-service-ca\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.222381 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.222357 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-oauth-serving-cert\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.222485 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.222463 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-trusted-ca-bundle\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.224020 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.223994 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-oauth-config\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.224108 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.224082 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-serving-cert\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.246728 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.246706 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gpgm\" (UniqueName: \"kubernetes.io/projected/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-kube-api-access-5gpgm\") pod \"console-7986965f75-j6r64\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.281047 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.281000 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:36.378114 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.378083 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:36.494042 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.494011 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:36.700009 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.698007 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7986965f75-j6r64"] Oct 30 19:48:36.702694 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:48:36.702663 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb398b2ac_7ae4_48a9_a61a_0257dbddbadc.slice/crio-171638db34be252f30927f2ffbcf3c3a303b217c53272fe6e0c2830d24f77086 WatchSource:0}: Error finding container 171638db34be252f30927f2ffbcf3c3a303b217c53272fe6e0c2830d24f77086: Status 404 returned error can't find the container with id 171638db34be252f30927f2ffbcf3c3a303b217c53272fe6e0c2830d24f77086 Oct 30 19:48:36.752021 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.751446 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:36.937242 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.937203 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7986965f75-j6r64" event={"ID":"b398b2ac-7ae4-48a9-a61a-0257dbddbadc","Type":"ContainerStarted","Data":"1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea"} Oct 30 19:48:36.937404 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.937263 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7986965f75-j6r64" event={"ID":"b398b2ac-7ae4-48a9-a61a-0257dbddbadc","Type":"ContainerStarted","Data":"171638db34be252f30927f2ffbcf3c3a303b217c53272fe6e0c2830d24f77086"} Oct 30 19:48:36.996670 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:36.996532 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7986965f75-j6r64" podStartSLOduration=0.996517627 podStartE2EDuration="996.517627ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:36.994194565 +0000 UTC m=+33.394366571" watchObservedRunningTime="2026-10-30 19:48:36.996517627 +0000 UTC m=+33.396689633" Oct 30 19:48:39.362235 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:39.362203 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:48:46.378812 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:46.378779 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:46.379230 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:46.378858 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:46.383487 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:46.383466 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:46.972855 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:46.972825 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:48:47.096054 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:48:47.096018 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f66d775b6-dq96j"] Oct 30 19:49:05.089426 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:49:05.089391 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:49:07.878923 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:07.878867 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:49:07.894506 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:07.894483 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:49:08.059774 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:08.059749 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 19:49:08.445790 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:08.445760 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:12.114873 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.114835 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6f66d775b6-dq96j" podUID="b569a9c0-5370-4ceb-9a82-90dad326ffad" containerName="console" containerID="cri-o://aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d" gracePeriod=15 Oct 30 19:49:12.370519 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.370467 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:49:12.501486 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501454 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca\") pod \"b569a9c0-5370-4ceb-9a82-90dad326ffad\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " Oct 30 19:49:12.501653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501511 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config\") pod \"b569a9c0-5370-4ceb-9a82-90dad326ffad\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " Oct 30 19:49:12.501653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501543 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp\") pod \"b569a9c0-5370-4ceb-9a82-90dad326ffad\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " Oct 30 19:49:12.501653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501577 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6\" (UniqueName: \"kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6\") pod \"b569a9c0-5370-4ceb-9a82-90dad326ffad\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " Oct 30 19:49:12.501653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501602 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config\") pod \"b569a9c0-5370-4ceb-9a82-90dad326ffad\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " Oct 30 19:49:12.501653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501622 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert\") pod \"b569a9c0-5370-4ceb-9a82-90dad326ffad\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " Oct 30 19:49:12.501653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501645 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert\") pod \"b569a9c0-5370-4ceb-9a82-90dad326ffad\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " Oct 30 19:49:12.501953 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501675 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle\") pod \"b569a9c0-5370-4ceb-9a82-90dad326ffad\" (UID: \"b569a9c0-5370-4ceb-9a82-90dad326ffad\") " Oct 30 19:49:12.501953 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501936 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca" pod "b569a9c0-5370-4ceb-9a82-90dad326ffad" (UID: "b569a9c0-5370-4ceb-9a82-90dad326ffad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:49:12.502060 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.501961 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config" pod "b569a9c0-5370-4ceb-9a82-90dad326ffad" (UID: "b569a9c0-5370-4ceb-9a82-90dad326ffad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:49:12.502317 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.502290 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle" pod "b569a9c0-5370-4ceb-9a82-90dad326ffad" (UID: "b569a9c0-5370-4ceb-9a82-90dad326ffad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:49:12.502425 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.502327 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert" pod "b569a9c0-5370-4ceb-9a82-90dad326ffad" (UID: "b569a9c0-5370-4ceb-9a82-90dad326ffad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:49:12.502425 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.502362 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp" pod "b569a9c0-5370-4ceb-9a82-90dad326ffad" (UID: "b569a9c0-5370-4ceb-9a82-90dad326ffad"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:49:12.504139 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.504113 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config" pod "b569a9c0-5370-4ceb-9a82-90dad326ffad" (UID: "b569a9c0-5370-4ceb-9a82-90dad326ffad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:49:12.504565 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.504543 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert" pod "b569a9c0-5370-4ceb-9a82-90dad326ffad" (UID: "b569a9c0-5370-4ceb-9a82-90dad326ffad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:49:12.504657 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.504584 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6" pod "b569a9c0-5370-4ceb-9a82-90dad326ffad" (UID: "b569a9c0-5370-4ceb-9a82-90dad326ffad"). InnerVolumeSpecName "kube-api-access-glwp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:49:12.602849 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.602817 6411 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-oauth-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:12.602849 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.602845 6411 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-oauth-serving-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:12.603036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.602860 6411 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-serving-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:12.603036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.602872 6411 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-trusted-ca-bundle\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:12.603036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.602884 6411 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:12.603036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.602918 6411 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b569a9c0-5370-4ceb-9a82-90dad326ffad-console-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:12.603036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.602931 6411 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b569a9c0-5370-4ceb-9a82-90dad326ffad-tmp\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:12.603036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:12.602942 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glwp6\" (UniqueName: \"kubernetes.io/projected/b569a9c0-5370-4ceb-9a82-90dad326ffad-kube-api-access-glwp6\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:13.060677 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.060647 6411 generic.go:356] "Generic (PLEG): container finished" podID="b569a9c0-5370-4ceb-9a82-90dad326ffad" containerID="aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d" exitCode=0 Oct 30 19:49:13.060827 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.060698 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f66d775b6-dq96j" event={"ID":"b569a9c0-5370-4ceb-9a82-90dad326ffad","Type":"ContainerDied","Data":"aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d"} Oct 30 19:49:13.060827 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.060718 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f66d775b6-dq96j" Oct 30 19:49:13.060827 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.060734 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f66d775b6-dq96j" event={"ID":"b569a9c0-5370-4ceb-9a82-90dad326ffad","Type":"ContainerDied","Data":"5bbe69d136aefa4fd0c16be97fc6a022036f7086640c1a70cbcf76a8bc98f030"} Oct 30 19:49:13.060827 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.060758 6411 scope.go:122] "RemoveContainer" containerID="aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d" Oct 30 19:49:13.069137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.069120 6411 scope.go:122] "RemoveContainer" containerID="aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d" Oct 30 19:49:13.069394 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:49:13.069377 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d\": container with ID starting with aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d not found: ID does not exist" containerID="aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d" Oct 30 19:49:13.069439 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.069404 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d"} err="failed to get container status \"aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d\": rpc error: code = NotFound desc = could not find container \"aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d\": container with ID starting with aa249c9112688b2bb58aaec5b8142be8dbefa0ae07508d17a71587e24596412d not found: ID does not exist" Oct 30 19:49:13.115300 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.115275 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f66d775b6-dq96j"] Oct 30 19:49:13.127842 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.127815 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f66d775b6-dq96j"] Oct 30 19:49:13.275915 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.275872 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:13.730657 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:13.730627 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b569a9c0-5370-4ceb-9a82-90dad326ffad" path="/var/lib/kubelet/pods/b569a9c0-5370-4ceb-9a82-90dad326ffad/volumes" Oct 30 19:49:15.884777 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:15.884736 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7749b9cc5c-vbb2b"] Oct 30 19:49:15.885273 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:15.885078 6411 cpu_manager.go:414] "containerMap: removing container" podUID="b569a9c0-5370-4ceb-9a82-90dad326ffad" containerName="console" Oct 30 19:49:15.885273 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:15.885097 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b569a9c0-5370-4ceb-9a82-90dad326ffad" containerName="console" Oct 30 19:49:15.885273 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:15.885191 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="b569a9c0-5370-4ceb-9a82-90dad326ffad" containerName="console" Oct 30 19:49:15.888080 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:15.888065 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:15.913865 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:15.913828 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7749b9cc5c-vbb2b"] Oct 30 19:49:16.028538 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.028497 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e68ea6be-6fec-4347-a3f9-720743a2eaf2-tmp\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.028729 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.028548 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-oauth-config\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.028729 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.028641 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-trusted-ca-bundle\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.028729 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.028673 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-oauth-serving-cert\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.028867 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.028745 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-serving-cert\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.028867 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.028778 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sc4r\" (UniqueName: \"kubernetes.io/projected/e68ea6be-6fec-4347-a3f9-720743a2eaf2-kube-api-access-5sc4r\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.028867 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.028803 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-config\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.028867 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.028820 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-service-ca\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.043033 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.043004 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:16.130137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130097 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e68ea6be-6fec-4347-a3f9-720743a2eaf2-tmp\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.130326 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130154 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-oauth-config\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.130326 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130175 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-trusted-ca-bundle\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.130326 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130190 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-oauth-serving-cert\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.130326 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130213 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-serving-cert\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.130326 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130255 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sc4r\" (UniqueName: \"kubernetes.io/projected/e68ea6be-6fec-4347-a3f9-720743a2eaf2-kube-api-access-5sc4r\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.130326 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130290 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-config\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.130326 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130315 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-service-ca\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.130773 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.130542 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e68ea6be-6fec-4347-a3f9-720743a2eaf2-tmp\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.131039 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.131017 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-trusted-ca-bundle\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.131129 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.131108 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-config\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.131397 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.131379 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-service-ca\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.131472 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.131408 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e68ea6be-6fec-4347-a3f9-720743a2eaf2-oauth-serving-cert\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.133755 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.133732 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-oauth-config\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.133836 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.133780 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e68ea6be-6fec-4347-a3f9-720743a2eaf2-console-serving-cert\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.160736 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.160674 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sc4r\" (UniqueName: \"kubernetes.io/projected/e68ea6be-6fec-4347-a3f9-720743a2eaf2-kube-api-access-5sc4r\") pod \"console-7749b9cc5c-vbb2b\" (UID: \"e68ea6be-6fec-4347-a3f9-720743a2eaf2\") " pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.186012 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.185985 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:16.198906 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.198875 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:16.413068 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.412995 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:16.497434 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.497404 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7749b9cc5c-vbb2b"] Oct 30 19:49:16.500677 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:49:16.500652 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68ea6be_6fec_4347_a3f9_720743a2eaf2.slice/crio-a2bfe6c1640e0f21452da5fa1dfaa66cb9f41614d6347e8ba10b3b48d2d1dd6c WatchSource:0}: Error finding container a2bfe6c1640e0f21452da5fa1dfaa66cb9f41614d6347e8ba10b3b48d2d1dd6c: Status 404 returned error can't find the container with id a2bfe6c1640e0f21452da5fa1dfaa66cb9f41614d6347e8ba10b3b48d2d1dd6c Oct 30 19:49:16.550384 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:16.550362 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:17.077948 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:17.077909 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7749b9cc5c-vbb2b" event={"ID":"e68ea6be-6fec-4347-a3f9-720743a2eaf2","Type":"ContainerStarted","Data":"155302c60f520329d863b095fe69fd75afdaed56e7501767cafbf9f6b16d4bde"} Oct 30 19:49:17.077948 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:17.077956 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7749b9cc5c-vbb2b" event={"ID":"e68ea6be-6fec-4347-a3f9-720743a2eaf2","Type":"ContainerStarted","Data":"a2bfe6c1640e0f21452da5fa1dfaa66cb9f41614d6347e8ba10b3b48d2d1dd6c"} Oct 30 19:49:17.129039 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:17.128994 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7749b9cc5c-vbb2b" podStartSLOduration=2.128980348 podStartE2EDuration="2.128980348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:17.127667454 +0000 UTC m=+73.527839460" watchObservedRunningTime="2026-10-30 19:49:17.128980348 +0000 UTC m=+73.529152352" Oct 30 19:49:25.988615 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:25.988580 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:26.130206 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:26.130168 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:26.199681 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:26.199660 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:26.199761 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:26.199751 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:26.204471 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:26.204450 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:27.114812 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:27.114786 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7749b9cc5c-vbb2b" Oct 30 19:49:27.195036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:27.195000 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7986965f75-j6r64"] Oct 30 19:49:32.918346 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:32.918310 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:33.076133 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:33.076100 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:35.090445 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:49:35.090407 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:49:43.063916 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:43.063823 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:43.187064 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:43.187032 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:52.214555 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.214483 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7986965f75-j6r64" podUID="b398b2ac-7ae4-48a9-a61a-0257dbddbadc" containerName="console" containerID="cri-o://1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea" gracePeriod=15 Oct 30 19:49:52.467596 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.467541 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:49:52.617404 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617365 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-oauth-serving-cert\") pod \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " Oct 30 19:49:52.617590 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617418 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-oauth-config\") pod \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " Oct 30 19:49:52.617590 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617447 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-config\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-config\") pod \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " Oct 30 19:49:52.617590 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617476 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-kube-api-access-5gpgm\" (UniqueName: \"kubernetes.io/projected/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-kube-api-access-5gpgm\") pod \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " Oct 30 19:49:52.617590 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617520 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-tmp\") pod \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " Oct 30 19:49:52.617590 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617552 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-service-ca\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-service-ca\") pod \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " Oct 30 19:49:52.617590 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617589 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-serving-cert\") pod \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " Oct 30 19:49:52.617833 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617629 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-trusted-ca-bundle\") pod \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\" (UID: \"b398b2ac-7ae4-48a9-a61a-0257dbddbadc\") " Oct 30 19:49:52.617955 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617930 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-oauth-serving-cert" pod "b398b2ac-7ae4-48a9-a61a-0257dbddbadc" (UID: "b398b2ac-7ae4-48a9-a61a-0257dbddbadc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:49:52.618029 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.617969 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-tmp" pod "b398b2ac-7ae4-48a9-a61a-0257dbddbadc" (UID: "b398b2ac-7ae4-48a9-a61a-0257dbddbadc"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:49:52.618080 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.618048 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-config" pod "b398b2ac-7ae4-48a9-a61a-0257dbddbadc" (UID: "b398b2ac-7ae4-48a9-a61a-0257dbddbadc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:49:52.618339 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.618237 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-trusted-ca-bundle" pod "b398b2ac-7ae4-48a9-a61a-0257dbddbadc" (UID: "b398b2ac-7ae4-48a9-a61a-0257dbddbadc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:49:52.618339 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.618324 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-service-ca" pod "b398b2ac-7ae4-48a9-a61a-0257dbddbadc" (UID: "b398b2ac-7ae4-48a9-a61a-0257dbddbadc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:49:52.619925 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.619885 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-serving-cert" pod "b398b2ac-7ae4-48a9-a61a-0257dbddbadc" (UID: "b398b2ac-7ae4-48a9-a61a-0257dbddbadc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:49:52.620018 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.619973 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-kube-api-access-5gpgm" pod "b398b2ac-7ae4-48a9-a61a-0257dbddbadc" (UID: "b398b2ac-7ae4-48a9-a61a-0257dbddbadc"). InnerVolumeSpecName "kube-api-access-5gpgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:49:52.620018 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.620006 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-oauth-config" pod "b398b2ac-7ae4-48a9-a61a-0257dbddbadc" (UID: "b398b2ac-7ae4-48a9-a61a-0257dbddbadc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:49:52.719071 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.719006 6411 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-tmp\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:52.719071 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.719029 6411 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:52.719071 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.719040 6411 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-serving-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:52.719071 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.719049 6411 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-trusted-ca-bundle\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:52.719071 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.719058 6411 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-oauth-serving-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:52.719071 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.719066 6411 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-oauth-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:52.719071 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.719075 6411 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-console-config\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:52.719389 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:52.719083 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gpgm\" (UniqueName: \"kubernetes.io/projected/b398b2ac-7ae4-48a9-a61a-0257dbddbadc-kube-api-access-5gpgm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:49:53.191962 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.191930 6411 generic.go:356] "Generic (PLEG): container finished" podID="b398b2ac-7ae4-48a9-a61a-0257dbddbadc" containerID="1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea" exitCode=0 Oct 30 19:49:53.192126 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.191966 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7986965f75-j6r64" event={"ID":"b398b2ac-7ae4-48a9-a61a-0257dbddbadc","Type":"ContainerDied","Data":"1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea"} Oct 30 19:49:53.192126 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.191976 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7986965f75-j6r64" Oct 30 19:49:53.192126 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.192010 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7986965f75-j6r64" event={"ID":"b398b2ac-7ae4-48a9-a61a-0257dbddbadc","Type":"ContainerDied","Data":"171638db34be252f30927f2ffbcf3c3a303b217c53272fe6e0c2830d24f77086"} Oct 30 19:49:53.192126 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.192029 6411 scope.go:122] "RemoveContainer" containerID="1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea" Oct 30 19:49:53.200786 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.200763 6411 scope.go:122] "RemoveContainer" containerID="1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea" Oct 30 19:49:53.201088 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:49:53.201069 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea\": container with ID starting with 1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea not found: ID does not exist" containerID="1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea" Oct 30 19:49:53.201172 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.201097 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea"} err="failed to get container status \"1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea\": rpc error: code = NotFound desc = could not find container \"1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea\": container with ID starting with 1d52ebd02f9f86d77af96f2b7f342900100f683a49c8d06740b656e4f8b93eea not found: ID does not exist" Oct 30 19:49:53.248136 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.248104 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7986965f75-j6r64"] Oct 30 19:49:53.260312 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.260285 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7986965f75-j6r64"] Oct 30 19:49:53.730922 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:49:53.730872 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b398b2ac-7ae4-48a9-a61a-0257dbddbadc" path="/var/lib/kubelet/pods/b398b2ac-7ae4-48a9-a61a-0257dbddbadc/volumes" Oct 30 19:50:05.089597 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:50:05.089550 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:50:08.553782 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:50:08.553747 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:50:35.089681 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:50:35.089640 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:50:51.495090 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:50:51.495051 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:50:51.622762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:50:51.622728 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:05.089766 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:51:05.089721 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:51:08.701036 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:08.701003 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:09.639014 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:09.638981 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:09.789143 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:09.789105 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:18.153639 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:18.153554 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:19.103972 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:19.103938 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:31.811267 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:31.811226 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:32.642604 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:32.642572 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:32.797029 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:32.796981 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:35.092083 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:51:35.092041 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:51:35.270454 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:35.270418 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:35.527863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:35.527828 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:36.546111 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:36.546080 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:36.766724 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:36.766685 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:37.003777 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:37.003753 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:37.262536 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:37.262458 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:39.417772 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:39.417735 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:39.624105 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:39.624072 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:42.035499 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:42.035421 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:44.122648 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:44.122619 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:46.632180 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:46.632143 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:47.049585 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.049506 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-289mx"] Oct 30 19:51:47.049789 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.049777 6411 cpu_manager.go:414] "containerMap: removing container" podUID="b398b2ac-7ae4-48a9-a61a-0257dbddbadc" containerName="console" Oct 30 19:51:47.049826 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.049790 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b398b2ac-7ae4-48a9-a61a-0257dbddbadc" containerName="console" Oct 30 19:51:47.049864 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.049835 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="b398b2ac-7ae4-48a9-a61a-0257dbddbadc" containerName="console" Oct 30 19:51:47.052535 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.052521 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-289mx" Oct 30 19:51:47.059039 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.059021 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/416a943d-2f63-4e49-994a-174a5ed3d297-bound-sa-token\") pod \"cert-manager-597b95cb84-289mx\" (UID: \"416a943d-2f63-4e49-994a-174a5ed3d297\") " pod="cert-manager/cert-manager-597b95cb84-289mx" Oct 30 19:51:47.059153 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.059057 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9cc\" (UniqueName: \"kubernetes.io/projected/416a943d-2f63-4e49-994a-174a5ed3d297-kube-api-access-nt9cc\") pod \"cert-manager-597b95cb84-289mx\" (UID: \"416a943d-2f63-4e49-994a-174a5ed3d297\") " pod="cert-manager/cert-manager-597b95cb84-289mx" Oct 30 19:51:47.084823 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.084797 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-289mx"] Oct 30 19:51:47.087580 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.087556 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4x2dq\"" Oct 30 19:51:47.087580 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.087559 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Oct 30 19:51:47.087721 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.087582 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Oct 30 19:51:47.160068 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.160045 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9cc\" (UniqueName: \"kubernetes.io/projected/416a943d-2f63-4e49-994a-174a5ed3d297-kube-api-access-nt9cc\") pod \"cert-manager-597b95cb84-289mx\" (UID: \"416a943d-2f63-4e49-994a-174a5ed3d297\") " pod="cert-manager/cert-manager-597b95cb84-289mx" Oct 30 19:51:47.160222 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.160138 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/416a943d-2f63-4e49-994a-174a5ed3d297-bound-sa-token\") pod \"cert-manager-597b95cb84-289mx\" (UID: \"416a943d-2f63-4e49-994a-174a5ed3d297\") " pod="cert-manager/cert-manager-597b95cb84-289mx" Oct 30 19:51:47.186434 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.186407 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/416a943d-2f63-4e49-994a-174a5ed3d297-bound-sa-token\") pod \"cert-manager-597b95cb84-289mx\" (UID: \"416a943d-2f63-4e49-994a-174a5ed3d297\") " pod="cert-manager/cert-manager-597b95cb84-289mx" Oct 30 19:51:47.186536 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.186515 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9cc\" (UniqueName: \"kubernetes.io/projected/416a943d-2f63-4e49-994a-174a5ed3d297-kube-api-access-nt9cc\") pod \"cert-manager-597b95cb84-289mx\" (UID: \"416a943d-2f63-4e49-994a-174a5ed3d297\") " pod="cert-manager/cert-manager-597b95cb84-289mx" Oct 30 19:51:47.377082 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.377001 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-289mx" Oct 30 19:51:47.672399 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.672367 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-289mx"] Oct 30 19:51:47.675797 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:51:47.675772 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod416a943d_2f63_4e49_994a_174a5ed3d297.slice/crio-3a83f8bfbf1f8f0c56547763dfab1ecc4e31c1447f5bf212de05797af6644df4 WatchSource:0}: Error finding container 3a83f8bfbf1f8f0c56547763dfab1ecc4e31c1447f5bf212de05797af6644df4: Status 404 returned error can't find the container with id 3a83f8bfbf1f8f0c56547763dfab1ecc4e31c1447f5bf212de05797af6644df4 Oct 30 19:51:47.677740 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:47.677720 6411 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:51:48.556467 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:48.556428 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-289mx" event={"ID":"416a943d-2f63-4e49-994a-174a5ed3d297","Type":"ContainerStarted","Data":"3a83f8bfbf1f8f0c56547763dfab1ecc4e31c1447f5bf212de05797af6644df4"} Oct 30 19:51:51.569633 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:51.569600 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-289mx" event={"ID":"416a943d-2f63-4e49-994a-174a5ed3d297","Type":"ContainerStarted","Data":"d948aaad074755b578d98ec76429f1ba6e639ee14fcff78295fa020a39c3b6ce"} Oct 30 19:51:51.613099 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:51.613058 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-289mx" podStartSLOduration=1.757237937 podStartE2EDuration="4.613045209s" totalImagesPullingTime="2.855807272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:51.612367542 +0000 UTC m=+228.012539549" watchObservedRunningTime="2026-10-30 19:51:51.613045209 +0000 UTC m=+228.013217218" Oct 30 19:51:53.660279 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:53.660247 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:53.784104 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:53.784073 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:56.999601 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:51:56.999571 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:01.711984 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:01.711952 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:01.927621 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:01.927582 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:03.438600 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:03.438565 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:05.092657 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:52:05.092616 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:52:06.668253 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:06.668222 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:06.820153 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:06.820111 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:08.907575 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:08.907531 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:11.947413 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:11.947383 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:12.169467 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.169431 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2"] Oct 30 19:52:12.172665 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.172646 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.193685 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.193661 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-74w9p\"" Oct 30 19:52:12.206681 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.206601 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Oct 30 19:52:12.206681 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.206632 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:52:12.206869 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.206752 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Oct 30 19:52:12.207374 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.207352 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2"] Oct 30 19:52:12.208171 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.208155 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Oct 30 19:52:12.236298 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.236273 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16956fe1-a88f-4925-9a7c-2ff97bacc353-cert\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.236414 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.236308 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/16956fe1-a88f-4925-9a7c-2ff97bacc353-metrics-cert\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.236463 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.236436 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9898\" (UniqueName: \"kubernetes.io/projected/16956fe1-a88f-4925-9a7c-2ff97bacc353-kube-api-access-h9898\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.236500 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.236460 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/16956fe1-a88f-4925-9a7c-2ff97bacc353-manager-config\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.254846 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.254822 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:12.299744 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.299720 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Oct 30 19:52:12.337181 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.337157 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9898\" (UniqueName: \"kubernetes.io/projected/16956fe1-a88f-4925-9a7c-2ff97bacc353-kube-api-access-h9898\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.337268 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.337195 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/16956fe1-a88f-4925-9a7c-2ff97bacc353-manager-config\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.337268 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.337222 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16956fe1-a88f-4925-9a7c-2ff97bacc353-cert\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.337268 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.337241 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/16956fe1-a88f-4925-9a7c-2ff97bacc353-metrics-cert\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.337817 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.337799 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/16956fe1-a88f-4925-9a7c-2ff97bacc353-manager-config\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.339760 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.339739 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16956fe1-a88f-4925-9a7c-2ff97bacc353-cert\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.339930 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.339886 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/16956fe1-a88f-4925-9a7c-2ff97bacc353-metrics-cert\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.363367 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.363336 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9898\" (UniqueName: \"kubernetes.io/projected/16956fe1-a88f-4925-9a7c-2ff97bacc353-kube-api-access-h9898\") pod \"lws-controller-manager-77f878f8c6-kn7q2\" (UID: \"16956fe1-a88f-4925-9a7c-2ff97bacc353\") " pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.483215 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.483158 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:12.789783 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:12.789604 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2"] Oct 30 19:52:12.792386 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:52:12.792358 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16956fe1_a88f_4925_9a7c_2ff97bacc353.slice/crio-75ae62dafc58bda6c73e8079689c0c40d356f2896b1f4dc514c58d3535d493ce WatchSource:0}: Error finding container 75ae62dafc58bda6c73e8079689c0c40d356f2896b1f4dc514c58d3535d493ce: Status 404 returned error can't find the container with id 75ae62dafc58bda6c73e8079689c0c40d356f2896b1f4dc514c58d3535d493ce Oct 30 19:52:13.641738 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:13.641701 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" event={"ID":"16956fe1-a88f-4925-9a7c-2ff97bacc353","Type":"ContainerStarted","Data":"75ae62dafc58bda6c73e8079689c0c40d356f2896b1f4dc514c58d3535d493ce"} Oct 30 19:52:13.647529 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:13.647505 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:13.824442 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:13.824412 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:14.303795 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:14.303763 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:14.498865 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:14.498833 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:14.719424 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:14.719386 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:15.176861 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.176828 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:15.403334 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.403307 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk"] Oct 30 19:52:15.406242 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.406224 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.448210 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.448148 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk"] Oct 30 19:52:15.462687 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.462665 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fc06a89-0b46-4c86-9271-4f2552ca4625-manifests\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.462811 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.462717 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fc06a89-0b46-4c86-9271-4f2552ca4625-tmp\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.462811 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.462741 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.462925 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.462874 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpm9m\" (UniqueName: \"kubernetes.io/projected/9fc06a89-0b46-4c86-9271-4f2552ca4625-kube-api-access-lpm9m\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.463026 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.463011 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-metrics-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.463087 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.463061 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.466690 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.466669 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 30 19:52:15.466787 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.466772 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 30 19:52:15.467337 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.467322 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\"" Oct 30 19:52:15.467390 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.467324 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4r48g\"" Oct 30 19:52:15.468084 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.468068 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 30 19:52:15.468259 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.468244 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 30 19:52:15.495237 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.495217 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:15.564468 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.564440 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fc06a89-0b46-4c86-9271-4f2552ca4625-manifests\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.564638 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.564481 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fc06a89-0b46-4c86-9271-4f2552ca4625-tmp\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.564638 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.564499 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.564638 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.564517 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpm9m\" (UniqueName: \"kubernetes.io/projected/9fc06a89-0b46-4c86-9271-4f2552ca4625-kube-api-access-lpm9m\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.564638 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.564549 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-metrics-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.564859 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.564650 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.564946 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.564867 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fc06a89-0b46-4c86-9271-4f2552ca4625-manifests\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.566945 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.566925 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fc06a89-0b46-4c86-9271-4f2552ca4625-tmp\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.567062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.566993 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.567266 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.567244 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-metrics-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.567328 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.567244 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fc06a89-0b46-4c86-9271-4f2552ca4625-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.598072 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.598051 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpm9m\" (UniqueName: \"kubernetes.io/projected/9fc06a89-0b46-4c86-9271-4f2552ca4625-kube-api-access-lpm9m\") pod \"opendatahub-operator-controller-manager-6f6c7b487f-7jznk\" (UID: \"9fc06a89-0b46-4c86-9271-4f2552ca4625\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.652526 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.652494 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" event={"ID":"16956fe1-a88f-4925-9a7c-2ff97bacc353","Type":"ContainerStarted","Data":"0698b4fed1cc49bbd77e9ac13d88d9f42f0d11b26f88dc508cfaef51ea34c450"} Oct 30 19:52:15.653123 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.653108 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:15.657514 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.657493 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:15.697993 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.697949 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" podStartSLOduration=1.019804596 podStartE2EDuration="3.697934721s" totalImagesPullingTime="2.678130125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:15.696709533 +0000 UTC m=+252.096881562" watchObservedRunningTime="2026-10-30 19:52:15.697934721 +0000 UTC m=+252.098106726" Oct 30 19:52:15.721078 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.721025 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:15.782447 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.782421 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:15.971255 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:15.971182 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:16.061694 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:16.061665 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk"] Oct 30 19:52:16.064974 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:52:16.064950 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc06a89_0b46_4c86_9271_4f2552ca4625.slice/crio-26478a61fa01c757cc6ebf4fbd5ed2acdbdc6a7e74465e18b990f61a54e23851 WatchSource:0}: Error finding container 26478a61fa01c757cc6ebf4fbd5ed2acdbdc6a7e74465e18b990f61a54e23851: Status 404 returned error can't find the container with id 26478a61fa01c757cc6ebf4fbd5ed2acdbdc6a7e74465e18b990f61a54e23851 Oct 30 19:52:16.149308 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:16.149275 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:16.659375 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:16.659332 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" event={"ID":"9fc06a89-0b46-4c86-9271-4f2552ca4625","Type":"ContainerStarted","Data":"26478a61fa01c757cc6ebf4fbd5ed2acdbdc6a7e74465e18b990f61a54e23851"} Oct 30 19:52:19.671498 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:19.671407 6411 generic.go:356] "Generic (PLEG): container finished" podID="9fc06a89-0b46-4c86-9271-4f2552ca4625" containerID="28cef20295b1871e7194a91a725a2d1fe318c5f7a8c0042f24b649b313e8f435" exitCode=0 Oct 30 19:52:19.671498 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:19.671459 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" event={"ID":"9fc06a89-0b46-4c86-9271-4f2552ca4625","Type":"ContainerDied","Data":"28cef20295b1871e7194a91a725a2d1fe318c5f7a8c0042f24b649b313e8f435"} Oct 30 19:52:20.676539 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:20.676499 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" event={"ID":"9fc06a89-0b46-4c86-9271-4f2552ca4625","Type":"ContainerStarted","Data":"cf8268a9f8e1cc9d60e7dfd8502c605b5550f018ba534e91f27b6bc0e305f8c5"} Oct 30 19:52:20.676936 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:20.676637 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:20.767765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:20.767714 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" podStartSLOduration=2.240965793 podStartE2EDuration="5.767681313s" totalImagesPullingTime="3.067216371s" totalInitContainerRuntime="459.499149ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:20.75469252 +0000 UTC m=+257.154864526" watchObservedRunningTime="2026-10-30 19:52:20.767681313 +0000 UTC m=+257.167853310" Oct 30 19:52:27.666396 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:27.666366 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77f878f8c6-kn7q2" Oct 30 19:52:31.682061 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:31.682028 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f6c7b487f-7jznk" Oct 30 19:52:35.089834 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:52:35.089797 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:52:55.354999 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.354951 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:55.817515 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.817482 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9"] Oct 30 19:52:55.827244 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.827216 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:55.844372 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.844337 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9"] Oct 30 19:52:55.855912 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.855874 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 19:52:55.856025 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.855876 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 19:52:55.884449 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.884421 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa5b7421-a7a7-4ea6-8d97-884649e01eac-tls-certs\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:55.884569 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.884453 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzlkg\" (UniqueName: \"kubernetes.io/projected/fa5b7421-a7a7-4ea6-8d97-884649e01eac-kube-api-access-qzlkg\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:55.884569 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.884470 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa5b7421-a7a7-4ea6-8d97-884649e01eac-tmp\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:55.932158 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.932135 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jxhc6\"" Oct 30 19:52:55.932276 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.932250 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Oct 30 19:52:55.932276 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.932248 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Oct 30 19:52:55.985684 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.985657 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa5b7421-a7a7-4ea6-8d97-884649e01eac-tls-certs\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:55.985807 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.985689 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlkg\" (UniqueName: \"kubernetes.io/projected/fa5b7421-a7a7-4ea6-8d97-884649e01eac-kube-api-access-qzlkg\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:55.985807 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.985715 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa5b7421-a7a7-4ea6-8d97-884649e01eac-tmp\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:55.988182 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.988156 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa5b7421-a7a7-4ea6-8d97-884649e01eac-tmp\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:55.988389 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:55.988371 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa5b7421-a7a7-4ea6-8d97-884649e01eac-tls-certs\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:56.017911 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:56.017868 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzlkg\" (UniqueName: \"kubernetes.io/projected/fa5b7421-a7a7-4ea6-8d97-884649e01eac-kube-api-access-qzlkg\") pod \"kube-auth-proxy-6f5c96996f-sjwt9\" (UID: \"fa5b7421-a7a7-4ea6-8d97-884649e01eac\") " pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:56.138791 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:56.138756 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" Oct 30 19:52:56.451925 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:56.451885 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9"] Oct 30 19:52:56.453388 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:52:56.453363 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5b7421_a7a7_4ea6_8d97_884649e01eac.slice/crio-5454c401c040974752a5d90a6566833e4363c5be8559dc8c62da127ac85b99a2 WatchSource:0}: Error finding container 5454c401c040974752a5d90a6566833e4363c5be8559dc8c62da127ac85b99a2: Status 404 returned error can't find the container with id 5454c401c040974752a5d90a6566833e4363c5be8559dc8c62da127ac85b99a2 Oct 30 19:52:56.635752 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:56.635718 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:56.801724 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:56.801652 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" event={"ID":"fa5b7421-a7a7-4ea6-8d97-884649e01eac","Type":"ContainerStarted","Data":"5454c401c040974752a5d90a6566833e4363c5be8559dc8c62da127ac85b99a2"} Oct 30 19:52:58.082458 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:58.082417 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:58.224528 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:58.224489 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:59.274757 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:59.274725 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:59.627111 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:59.627016 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:59.774491 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:59.774457 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:52:59.814576 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:59.814549 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" event={"ID":"fa5b7421-a7a7-4ea6-8d97-884649e01eac","Type":"ContainerStarted","Data":"cad4ead4f99e9875a15595cefc858c4bb850138ab6e2862a433e584dadd95b02"} Oct 30 19:52:59.872713 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:52:59.872672 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f5c96996f-sjwt9" podStartSLOduration=2.511639126 podStartE2EDuration="4.872657305s" totalImagesPullingTime="2.361018179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:59.870125512 +0000 UTC m=+296.270297519" watchObservedRunningTime="2026-10-30 19:52:59.872657305 +0000 UTC m=+296.272829364" Oct 30 19:53:00.078644 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:00.078611 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:01.680782 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:01.680749 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:01.814437 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:01.814389 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:02.825727 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:02.825677 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:02.951809 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:02.951776 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:03.200517 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:03.200485 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:05.092360 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:53:05.092306 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:53:08.945255 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:08.945208 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:23.512478 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:23.512440 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:23.682045 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:23.682012 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:35.089938 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:53:35.089878 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:53:57.564004 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:57.563972 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:53:58.477883 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:53:58.477830 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:05.089580 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:54:05.089541 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:54:09.274433 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:09.274400 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:11.861527 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:11.861428 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:12.080778 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.080743 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:12.384198 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.383718 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-h8mrc"] Oct 30 19:54:12.389409 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.389385 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.398671 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.398645 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:12.475882 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.475786 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Oct 30 19:54:12.475882 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.475788 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xx6c4\"" Oct 30 19:54:12.476225 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.476203 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\"" Oct 30 19:54:12.477332 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.477311 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-h8mrc"] Oct 30 19:54:12.480389 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.480366 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zzhm\" (UniqueName: \"kubernetes.io/projected/89171348-f79f-4eb0-b8be-b73a4130dc09-kube-api-access-8zzhm\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.480489 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.480410 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89171348-f79f-4eb0-b8be-b73a4130dc09-cert\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.480489 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.480431 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89171348-f79f-4eb0-b8be-b73a4130dc09-metrics-certs\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.581537 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.581510 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zzhm\" (UniqueName: \"kubernetes.io/projected/89171348-f79f-4eb0-b8be-b73a4130dc09-kube-api-access-8zzhm\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.581681 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.581556 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89171348-f79f-4eb0-b8be-b73a4130dc09-cert\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.581681 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.581576 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89171348-f79f-4eb0-b8be-b73a4130dc09-metrics-certs\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.584112 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.584090 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89171348-f79f-4eb0-b8be-b73a4130dc09-cert\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.584211 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.584132 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89171348-f79f-4eb0-b8be-b73a4130dc09-metrics-certs\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.620960 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.620934 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zzhm\" (UniqueName: \"kubernetes.io/projected/89171348-f79f-4eb0-b8be-b73a4130dc09-kube-api-access-8zzhm\") pod \"odh-model-controller-f87fcf56c-h8mrc\" (UID: \"89171348-f79f-4eb0-b8be-b73a4130dc09\") " pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.702009 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.701939 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:12.997577 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:12.997508 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-h8mrc"] Oct 30 19:54:13.000642 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:54:13.000608 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89171348_f79f_4eb0_b8be_b73a4130dc09.slice/crio-df17ee476f36a7c28a87ba539d5998c46208a0d40c62c91598e5bd8b50f49cba WatchSource:0}: Error finding container df17ee476f36a7c28a87ba539d5998c46208a0d40c62c91598e5bd8b50f49cba: Status 404 returned error can't find the container with id df17ee476f36a7c28a87ba539d5998c46208a0d40c62c91598e5bd8b50f49cba Oct 30 19:54:13.024390 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:13.024364 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:13.066122 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:13.066095 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" event={"ID":"89171348-f79f-4eb0-b8be-b73a4130dc09","Type":"ContainerStarted","Data":"df17ee476f36a7c28a87ba539d5998c46208a0d40c62c91598e5bd8b50f49cba"} Oct 30 19:54:15.403807 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:15.403770 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:15.689551 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:15.689457 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:15.866081 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:15.866047 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:54:16.081125 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:16.081050 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" event={"ID":"89171348-f79f-4eb0-b8be-b73a4130dc09","Type":"ContainerStarted","Data":"c41b532ab42d4efe02b34ce64a468d7ae26490d85e5e17dd6a62e989e36e287e"} Oct 30 19:54:16.081266 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:16.081183 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:16.134513 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:16.134456 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" podStartSLOduration=1.577872432 podStartE2EDuration="4.134437916s" totalImagesPullingTime="2.556565484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:54:16.13399084 +0000 UTC m=+372.534162861" watchObservedRunningTime="2026-10-30 19:54:16.134437916 +0000 UTC m=+372.534609925" Oct 30 19:54:27.087518 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:54:27.087487 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-f87fcf56c-h8mrc" Oct 30 19:54:35.090613 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:54:35.090572 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:55:05.089318 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:55:05.089281 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:55:08.872719 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:08.872685 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:09.034654 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:09.034618 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:09.222173 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:09.222136 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:21.059060 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:21.059025 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:23.980427 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:23.980394 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:25.923835 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:25.923796 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:26.484105 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:26.484076 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:26.773575 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:26.773487 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:28.723949 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:28.723914 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:30.098446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:30.098410 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:30.264119 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:30.264086 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:30.693936 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:30.693861 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:30.936942 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:30.936860 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:31.336742 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:31.336697 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:31.577619 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:31.577585 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:35.091690 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:55:35.091649 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:55:35.578962 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.578930 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj"] Oct 30 19:55:35.583719 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.583690 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:35.618313 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.618285 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 19:55:35.618579 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.618561 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 19:55:35.618999 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.618977 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5pgqn\"" Oct 30 19:55:35.619193 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.619174 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj"] Oct 30 19:55:35.687562 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.687527 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" (UID: \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:35.687770 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.687571 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cw5l\" (UniqueName: \"kubernetes.io/projected/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-kube-api-access-4cw5l\") pod \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" (UID: \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:35.788630 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.788588 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" (UID: \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:35.788802 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.788643 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw5l\" (UniqueName: \"kubernetes.io/projected/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-kube-api-access-4cw5l\") pod \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" (UID: \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:35.789015 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.788997 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" (UID: \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:35.814732 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.814697 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cw5l\" (UniqueName: \"kubernetes.io/projected/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-kube-api-access-4cw5l\") pod \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" (UID: \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:35.897025 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:35.896995 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:36.225635 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:36.225552 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj"] Oct 30 19:55:36.238652 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:55:36.238624 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7eba63_3f9b_46b0_98bf_f8022e5ce2e0.slice/crio-6058dabd94a8548916ae36ad0e7d1fbb2f8e594400ffd10f3d0143508016d392 WatchSource:0}: Error finding container 6058dabd94a8548916ae36ad0e7d1fbb2f8e594400ffd10f3d0143508016d392: Status 404 returned error can't find the container with id 6058dabd94a8548916ae36ad0e7d1fbb2f8e594400ffd10f3d0143508016d392 Oct 30 19:55:36.362559 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:36.362520 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" event={"ID":"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0","Type":"ContainerStarted","Data":"6058dabd94a8548916ae36ad0e7d1fbb2f8e594400ffd10f3d0143508016d392"} Oct 30 19:55:37.504544 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:37.504512 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:37.653025 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:37.652987 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:40.384724 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:40.384686 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" event={"ID":"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0","Type":"ContainerStarted","Data":"c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8"} Oct 30 19:55:40.385143 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:40.384863 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:40.467887 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:40.467785 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" podStartSLOduration=1.569891728 podStartE2EDuration="5.467772269s" totalImagesPullingTime="3.897880541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:55:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:55:40.467607531 +0000 UTC m=+456.867779538" watchObservedRunningTime="2026-10-30 19:55:40.467772269 +0000 UTC m=+456.867944277" Oct 30 19:55:40.598638 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:40.598599 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:40.817582 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:40.817507 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:43.485832 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:43.485802 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:43.705708 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:43.705670 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:50.190576 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:50.190542 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:51.393403 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:51.393368 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:52.474060 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:52.474028 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:52.753171 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:52.753058 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:52.967267 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:52.967213 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:55.361228 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.361195 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj"] Oct 30 19:55:55.361625 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.361402 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" podUID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" containerName="manager" containerID="cri-o://c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8" gracePeriod=2 Oct 30 19:55:55.365473 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.365444 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj"] Oct 30 19:55:55.376681 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:55:55.376636 6411 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" is forbidden: User \"system:node:ip-10-0-142-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-168.ec2.internal' and this object" podUID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:55.388504 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.388479 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z"] Oct 30 19:55:55.388768 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.388754 6411 cpu_manager.go:414] "containerMap: removing container" podUID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" containerName="manager" Oct 30 19:55:55.388768 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.388768 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" containerName="manager" Oct 30 19:55:55.388924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.388829 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" containerName="manager" Oct 30 19:55:55.391649 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.391630 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:55.405677 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:55:55.405651 6411 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" is forbidden: User \"system:node:ip-10-0-142-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-168.ec2.internal' and this object" podUID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:55.432521 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.432491 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z"] Oct 30 19:55:55.449394 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.449373 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a2d553c-7991-466b-9a06-947ca6d5dbbf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-rpm9z\" (UID: \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:55.449486 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.449409 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pktc\" (UniqueName: \"kubernetes.io/projected/5a2d553c-7991-466b-9a06-947ca6d5dbbf-kube-api-access-7pktc\") pod \"kuadrant-operator-controller-manager-6b6fb56846-rpm9z\" (UID: \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:55.550420 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.550391 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a2d553c-7991-466b-9a06-947ca6d5dbbf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-rpm9z\" (UID: \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:55.550540 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.550452 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pktc\" (UniqueName: \"kubernetes.io/projected/5a2d553c-7991-466b-9a06-947ca6d5dbbf-kube-api-access-7pktc\") pod \"kuadrant-operator-controller-manager-6b6fb56846-rpm9z\" (UID: \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:55.550883 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.550856 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a2d553c-7991-466b-9a06-947ca6d5dbbf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-rpm9z\" (UID: \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:55.577246 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.577216 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pktc\" (UniqueName: \"kubernetes.io/projected/5a2d553c-7991-466b-9a06-947ca6d5dbbf-kube-api-access-7pktc\") pod \"kuadrant-operator-controller-manager-6b6fb56846-rpm9z\" (UID: \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:55.618704 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.618659 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:55.632685 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:55:55.632659 6411 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-vwpnj\" is forbidden: User \"system:node:ip-10-0-142-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-168.ec2.internal' and this object" podUID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:55.651223 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.651202 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-extensions-socket-volume\") pod \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\" (UID: \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\") " Oct 30 19:55:55.651319 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.651235 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-kube-api-access-4cw5l\" (UniqueName: \"kubernetes.io/projected/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-kube-api-access-4cw5l\") pod \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\" (UID: \"8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0\") " Oct 30 19:55:55.651675 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.651646 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-extensions-socket-volume" pod "8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" (UID: "8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:55:55.653485 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.653467 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-kube-api-access-4cw5l" pod "8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" (UID: "8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0"). InnerVolumeSpecName "kube-api-access-4cw5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:55:55.731041 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.731014 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" path="/var/lib/kubelet/pods/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0/volumes" Oct 30 19:55:55.751836 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.751816 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:55.752058 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.752040 6411 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-extensions-socket-volume\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:55:55.752118 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:55.752060 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cw5l\" (UniqueName: \"kubernetes.io/projected/8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0-kube-api-access-4cw5l\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:55:56.092164 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.092132 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z"] Oct 30 19:55:56.120203 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:55:56.120177 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2d553c_7991_466b_9a06_947ca6d5dbbf.slice/crio-1aa18461ccf3b21735cb5a3880460123e3351213128efa1b2af0e5306ce29b9d WatchSource:0}: Error finding container 1aa18461ccf3b21735cb5a3880460123e3351213128efa1b2af0e5306ce29b9d: Status 404 returned error can't find the container with id 1aa18461ccf3b21735cb5a3880460123e3351213128efa1b2af0e5306ce29b9d Oct 30 19:55:56.452242 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.452209 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" event={"ID":"5a2d553c-7991-466b-9a06-947ca6d5dbbf","Type":"ContainerStarted","Data":"d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7"} Oct 30 19:55:56.452731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.452269 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" event={"ID":"5a2d553c-7991-466b-9a06-947ca6d5dbbf","Type":"ContainerStarted","Data":"1aa18461ccf3b21735cb5a3880460123e3351213128efa1b2af0e5306ce29b9d"} Oct 30 19:55:56.452731 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.452303 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:55:56.453729 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.453700 6411 generic.go:356] "Generic (PLEG): container finished" podID="8a7eba63-3f9b-46b0-98bf-f8022e5ce2e0" containerID="c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8" exitCode=0 Oct 30 19:55:56.453825 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.453740 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-vwpnj" Oct 30 19:55:56.453825 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.453789 6411 scope.go:122] "RemoveContainer" containerID="c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8" Oct 30 19:55:56.462056 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.462038 6411 scope.go:122] "RemoveContainer" containerID="c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8" Oct 30 19:55:56.462335 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:55:56.462314 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8\": container with ID starting with c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8 not found: ID does not exist" containerID="c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8" Oct 30 19:55:56.462409 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:56.462346 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8"} err="failed to get container status \"c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8\": rpc error: code = NotFound desc = could not find container \"c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8\": container with ID starting with c5c46f85552120e29430b145d29783f58df4b5f494e6f7bced9b6d44122179e8 not found: ID does not exist" Oct 30 19:55:59.717267 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:59.717231 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:55:59.894140 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:55:59.894108 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:01.313206 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:01.313165 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:01.456915 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:01.456860 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:05.090736 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:56:05.090704 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:56:07.461714 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:07.461680 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:56:07.477621 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:07.477590 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:07.535754 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:07.535700 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" podStartSLOduration=12.535683109 podStartE2EDuration="12.535683109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:55:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:55:56.505283513 +0000 UTC m=+472.905455543" watchObservedRunningTime="2026-10-30 19:56:07.535683109 +0000 UTC m=+483.935855116" Oct 30 19:56:07.642450 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:07.642415 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:09.325208 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:09.325173 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:10.447859 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:10.447816 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:10.594626 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:10.594598 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:10.814799 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:10.814725 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z"] Oct 30 19:56:10.814992 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:10.814945 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" podUID="5a2d553c-7991-466b-9a06-947ca6d5dbbf" containerName="manager" containerID="cri-o://d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7" gracePeriod=10 Oct 30 19:56:11.052838 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.052814 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h"] Oct 30 19:56:11.056081 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.056065 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:11.062401 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.062382 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:56:11.082486 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.082426 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h"] Oct 30 19:56:11.170364 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.170338 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a2d553c-7991-466b-9a06-947ca6d5dbbf-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a2d553c-7991-466b-9a06-947ca6d5dbbf-extensions-socket-volume\") pod \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\" (UID: \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\") " Oct 30 19:56:11.170558 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.170416 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a2d553c-7991-466b-9a06-947ca6d5dbbf-kube-api-access-7pktc\" (UniqueName: \"kubernetes.io/projected/5a2d553c-7991-466b-9a06-947ca6d5dbbf-kube-api-access-7pktc\") pod \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\" (UID: \"5a2d553c-7991-466b-9a06-947ca6d5dbbf\") " Oct 30 19:56:11.170634 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.170604 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88050ac8-8997-4c0c-8f10-4e3d3452a740-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vn25h\" (UID: \"88050ac8-8997-4c0c-8f10-4e3d3452a740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:11.170693 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.170676 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg6nz\" (UniqueName: \"kubernetes.io/projected/88050ac8-8997-4c0c-8f10-4e3d3452a740-kube-api-access-lg6nz\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vn25h\" (UID: \"88050ac8-8997-4c0c-8f10-4e3d3452a740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:11.170778 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.170759 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2d553c-7991-466b-9a06-947ca6d5dbbf-extensions-socket-volume" pod "5a2d553c-7991-466b-9a06-947ca6d5dbbf" (UID: "5a2d553c-7991-466b-9a06-947ca6d5dbbf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:56:11.172671 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.172643 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a2d553c-7991-466b-9a06-947ca6d5dbbf-kube-api-access-7pktc" pod "5a2d553c-7991-466b-9a06-947ca6d5dbbf" (UID: "5a2d553c-7991-466b-9a06-947ca6d5dbbf"). InnerVolumeSpecName "kube-api-access-7pktc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:56:11.271862 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.271828 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg6nz\" (UniqueName: \"kubernetes.io/projected/88050ac8-8997-4c0c-8f10-4e3d3452a740-kube-api-access-lg6nz\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vn25h\" (UID: \"88050ac8-8997-4c0c-8f10-4e3d3452a740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:11.272068 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.271937 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88050ac8-8997-4c0c-8f10-4e3d3452a740-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vn25h\" (UID: \"88050ac8-8997-4c0c-8f10-4e3d3452a740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:11.272068 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.271994 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pktc\" (UniqueName: \"kubernetes.io/projected/5a2d553c-7991-466b-9a06-947ca6d5dbbf-kube-api-access-7pktc\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:56:11.272068 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.272011 6411 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a2d553c-7991-466b-9a06-947ca6d5dbbf-extensions-socket-volume\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:56:11.272346 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.272326 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88050ac8-8997-4c0c-8f10-4e3d3452a740-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vn25h\" (UID: \"88050ac8-8997-4c0c-8f10-4e3d3452a740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:11.301424 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.301392 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg6nz\" (UniqueName: \"kubernetes.io/projected/88050ac8-8997-4c0c-8f10-4e3d3452a740-kube-api-access-lg6nz\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vn25h\" (UID: \"88050ac8-8997-4c0c-8f10-4e3d3452a740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:11.370316 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.370226 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:11.511919 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.511861 6411 generic.go:356] "Generic (PLEG): container finished" podID="5a2d553c-7991-466b-9a06-947ca6d5dbbf" containerID="d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7" exitCode=0 Oct 30 19:56:11.512427 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.511939 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" Oct 30 19:56:11.512427 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.511958 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" event={"ID":"5a2d553c-7991-466b-9a06-947ca6d5dbbf","Type":"ContainerDied","Data":"d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7"} Oct 30 19:56:11.512427 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.512019 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z" event={"ID":"5a2d553c-7991-466b-9a06-947ca6d5dbbf","Type":"ContainerDied","Data":"1aa18461ccf3b21735cb5a3880460123e3351213128efa1b2af0e5306ce29b9d"} Oct 30 19:56:11.512427 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.512088 6411 scope.go:122] "RemoveContainer" containerID="d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7" Oct 30 19:56:11.520709 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.520689 6411 scope.go:122] "RemoveContainer" containerID="d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7" Oct 30 19:56:11.521081 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:56:11.521059 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7\": container with ID starting with d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7 not found: ID does not exist" containerID="d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7" Oct 30 19:56:11.521128 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.521093 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7"} err="failed to get container status \"d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7\": rpc error: code = NotFound desc = could not find container \"d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7\": container with ID starting with d414cd9c79df3d411cdf838635dac728460ad8e5d42a3282e8ae3d5041121ee7 not found: ID does not exist" Oct 30 19:56:11.571161 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.571133 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z"] Oct 30 19:56:11.583719 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.583697 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-rpm9z"] Oct 30 19:56:11.682675 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.682638 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h"] Oct 30 19:56:11.686464 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:56:11.686433 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88050ac8_8997_4c0c_8f10_4e3d3452a740.slice/crio-b37181cf46d55a77659616d552fc05dcd2eedcb9d43a4422301068ab8f59157c WatchSource:0}: Error finding container b37181cf46d55a77659616d552fc05dcd2eedcb9d43a4422301068ab8f59157c: Status 404 returned error can't find the container with id b37181cf46d55a77659616d552fc05dcd2eedcb9d43a4422301068ab8f59157c Oct 30 19:56:11.732165 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:11.732138 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a2d553c-7991-466b-9a06-947ca6d5dbbf" path="/var/lib/kubelet/pods/5a2d553c-7991-466b-9a06-947ca6d5dbbf/volumes" Oct 30 19:56:12.403061 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:12.403018 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:12.518130 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:12.518082 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" event={"ID":"88050ac8-8997-4c0c-8f10-4e3d3452a740","Type":"ContainerStarted","Data":"9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177"} Oct 30 19:56:12.518544 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:12.518148 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" event={"ID":"88050ac8-8997-4c0c-8f10-4e3d3452a740","Type":"ContainerStarted","Data":"b37181cf46d55a77659616d552fc05dcd2eedcb9d43a4422301068ab8f59157c"} Oct 30 19:56:12.518544 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:12.518176 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:12.582340 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:12.582265 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" podStartSLOduration=1.582251715 podStartE2EDuration="1.582251715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:56:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:56:12.581790201 +0000 UTC m=+488.981962218" watchObservedRunningTime="2026-10-30 19:56:12.582251715 +0000 UTC m=+488.982423702" Oct 30 19:56:12.589555 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:12.589521 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:12.755787 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:12.755697 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:19.032198 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:19.032158 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:19.192658 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:19.192628 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:20.870363 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:20.870316 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:20.998446 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:20.998408 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:21.796012 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:21.795976 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:21.968961 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:21.968923 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:23.526307 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:23.526277 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 19:56:27.371510 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:27.371475 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:27.514004 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:27.513971 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:27.968242 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:27.968202 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:28.115824 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:28.115791 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:34.356364 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:34.356327 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:34.504841 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:34.504805 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:34.949123 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:34.949092 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:35.069112 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:35.069076 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:35.090951 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:56:35.090909 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:56:35.563058 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:35.563024 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:35.690169 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:35.690136 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:38.157007 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:38.156972 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:38.288307 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:38.288263 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:43.546138 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:43.546101 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:43.666449 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:43.666420 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:43.785804 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:43.785770 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:44.447794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.447766 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4v9m4"] Oct 30 19:56:44.448062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.448050 6411 cpu_manager.go:414] "containerMap: removing container" podUID="5a2d553c-7991-466b-9a06-947ca6d5dbbf" containerName="manager" Oct 30 19:56:44.448111 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.448064 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a2d553c-7991-466b-9a06-947ca6d5dbbf" containerName="manager" Oct 30 19:56:44.448145 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.448131 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a2d553c-7991-466b-9a06-947ca6d5dbbf" containerName="manager" Oct 30 19:56:44.451069 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.451047 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" Oct 30 19:56:44.472147 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.472128 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fznln\"" Oct 30 19:56:44.480403 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.480381 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4v9m4"] Oct 30 19:56:44.546143 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.546109 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lqbp\" (UniqueName: \"kubernetes.io/projected/525f38bd-dfe7-496e-87ed-f76d741bee03-kube-api-access-5lqbp\") pod \"authorino-57f7f8659f-4v9m4\" (UID: \"525f38bd-dfe7-496e-87ed-f76d741bee03\") " pod="kuadrant-system/authorino-57f7f8659f-4v9m4" Oct 30 19:56:44.591690 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.591657 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:44.647255 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.647226 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lqbp\" (UniqueName: \"kubernetes.io/projected/525f38bd-dfe7-496e-87ed-f76d741bee03-kube-api-access-5lqbp\") pod \"authorino-57f7f8659f-4v9m4\" (UID: \"525f38bd-dfe7-496e-87ed-f76d741bee03\") " pod="kuadrant-system/authorino-57f7f8659f-4v9m4" Oct 30 19:56:44.673794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.673773 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lqbp\" (UniqueName: \"kubernetes.io/projected/525f38bd-dfe7-496e-87ed-f76d741bee03-kube-api-access-5lqbp\") pod \"authorino-57f7f8659f-4v9m4\" (UID: \"525f38bd-dfe7-496e-87ed-f76d741bee03\") " pod="kuadrant-system/authorino-57f7f8659f-4v9m4" Oct 30 19:56:44.738297 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.738237 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:56:44.762513 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:44.762492 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" Oct 30 19:56:45.053336 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:45.053260 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4v9m4"] Oct 30 19:56:45.056285 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:56:45.056253 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod525f38bd_dfe7_496e_87ed_f76d741bee03.slice/crio-8f51b6f630645f2f69cdcddc333fd6ed521d994843f6bf3742d81af24357cfdd WatchSource:0}: Error finding container 8f51b6f630645f2f69cdcddc333fd6ed521d994843f6bf3742d81af24357cfdd: Status 404 returned error can't find the container with id 8f51b6f630645f2f69cdcddc333fd6ed521d994843f6bf3742d81af24357cfdd Oct 30 19:56:45.644857 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:45.644821 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" event={"ID":"525f38bd-dfe7-496e-87ed-f76d741bee03","Type":"ContainerStarted","Data":"8f51b6f630645f2f69cdcddc333fd6ed521d994843f6bf3742d81af24357cfdd"} Oct 30 19:56:47.654967 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:47.654928 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" event={"ID":"525f38bd-dfe7-496e-87ed-f76d741bee03","Type":"ContainerStarted","Data":"807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb"} Oct 30 19:56:47.696374 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:47.696314 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" podStartSLOduration=1.821712164 podStartE2EDuration="3.696296427s" totalImagesPullingTime="1.874584263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:56:47.695135524 +0000 UTC m=+524.095307532" watchObservedRunningTime="2026-10-30 19:56:47.696296427 +0000 UTC m=+524.096468433" Oct 30 19:56:49.013115 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:49.013080 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4v9m4"] Oct 30 19:56:49.663479 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:49.663420 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" podUID="525f38bd-dfe7-496e-87ed-f76d741bee03" containerName="authorino" containerID="cri-o://807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb" gracePeriod=30 Oct 30 19:56:49.901277 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:49.901256 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" Oct 30 19:56:49.991328 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:49.991261 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/525f38bd-dfe7-496e-87ed-f76d741bee03-kube-api-access-5lqbp\" (UniqueName: \"kubernetes.io/projected/525f38bd-dfe7-496e-87ed-f76d741bee03-kube-api-access-5lqbp\") pod \"525f38bd-dfe7-496e-87ed-f76d741bee03\" (UID: \"525f38bd-dfe7-496e-87ed-f76d741bee03\") " Oct 30 19:56:49.993331 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:49.993304 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525f38bd-dfe7-496e-87ed-f76d741bee03-kube-api-access-5lqbp" pod "525f38bd-dfe7-496e-87ed-f76d741bee03" (UID: "525f38bd-dfe7-496e-87ed-f76d741bee03"). InnerVolumeSpecName "kube-api-access-5lqbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:56:50.092182 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.092158 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lqbp\" (UniqueName: \"kubernetes.io/projected/525f38bd-dfe7-496e-87ed-f76d741bee03-kube-api-access-5lqbp\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:56:50.668735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.668699 6411 generic.go:356] "Generic (PLEG): container finished" podID="525f38bd-dfe7-496e-87ed-f76d741bee03" containerID="807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb" exitCode=0 Oct 30 19:56:50.668735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.668736 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" Oct 30 19:56:50.670967 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.668787 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" event={"ID":"525f38bd-dfe7-496e-87ed-f76d741bee03","Type":"ContainerDied","Data":"807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb"} Oct 30 19:56:50.670967 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.668824 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-4v9m4" event={"ID":"525f38bd-dfe7-496e-87ed-f76d741bee03","Type":"ContainerDied","Data":"8f51b6f630645f2f69cdcddc333fd6ed521d994843f6bf3742d81af24357cfdd"} Oct 30 19:56:50.670967 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.668842 6411 scope.go:122] "RemoveContainer" containerID="807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb" Oct 30 19:56:50.677083 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.677058 6411 scope.go:122] "RemoveContainer" containerID="807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb" Oct 30 19:56:50.677340 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:56:50.677322 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb\": container with ID starting with 807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb not found: ID does not exist" containerID="807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb" Oct 30 19:56:50.677396 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.677350 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb"} err="failed to get container status \"807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb\": rpc error: code = NotFound desc = could not find container \"807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb\": container with ID starting with 807f5f982173c70374bc206b18f744200221cac1f626e465237d0b923b6f6fbb not found: ID does not exist" Oct 30 19:56:50.723746 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.723715 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4v9m4"] Oct 30 19:56:50.729352 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:50.729324 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4v9m4"] Oct 30 19:56:51.730105 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:56:51.730074 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="525f38bd-dfe7-496e-87ed-f76d741bee03" path="/var/lib/kubelet/pods/525f38bd-dfe7-496e-87ed-f76d741bee03/volumes" Oct 30 19:57:02.740509 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:02.740461 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:02.894445 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:02.894410 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:03.530752 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:03.530717 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:05.091381 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:57:05.091344 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:57:09.446696 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:09.446647 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:15.596977 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.596889 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:15.724090 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.724041 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:15.906700 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.906669 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-hb24p"] Oct 30 19:57:15.907065 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.907047 6411 cpu_manager.go:414] "containerMap: removing container" podUID="525f38bd-dfe7-496e-87ed-f76d741bee03" containerName="authorino" Oct 30 19:57:15.907168 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.907067 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="525f38bd-dfe7-496e-87ed-f76d741bee03" containerName="authorino" Oct 30 19:57:15.907168 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.907151 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="525f38bd-dfe7-496e-87ed-f76d741bee03" containerName="authorino" Oct 30 19:57:15.910028 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.910008 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-hb24p" Oct 30 19:57:15.935400 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.935377 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fznln\"" Oct 30 19:57:15.936413 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.936391 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-hb24p"] Oct 30 19:57:15.987987 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:15.987957 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2dnz\" (UniqueName: \"kubernetes.io/projected/891098a4-9fe0-4f6b-a8d4-2460a99b79d3-kube-api-access-b2dnz\") pod \"authorino-6f6944bcdd-hb24p\" (UID: \"891098a4-9fe0-4f6b-a8d4-2460a99b79d3\") " pod="kuadrant-system/authorino-6f6944bcdd-hb24p" Oct 30 19:57:16.058144 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.058118 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-hb24p"] Oct 30 19:57:16.058299 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:57:16.058282 6411 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-b2dnz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f6944bcdd-hb24p" podUID="891098a4-9fe0-4f6b-a8d4-2460a99b79d3" Oct 30 19:57:16.088752 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.088729 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2dnz\" (UniqueName: \"kubernetes.io/projected/891098a4-9fe0-4f6b-a8d4-2460a99b79d3-kube-api-access-b2dnz\") pod \"authorino-6f6944bcdd-hb24p\" (UID: \"891098a4-9fe0-4f6b-a8d4-2460a99b79d3\") " pod="kuadrant-system/authorino-6f6944bcdd-hb24p" Oct 30 19:57:16.095613 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.095591 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-49xvf"] Oct 30 19:57:16.098675 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.098661 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.124769 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.124748 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-49xvf"] Oct 30 19:57:16.128357 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.128339 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Oct 30 19:57:16.129976 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.129958 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2dnz\" (UniqueName: \"kubernetes.io/projected/891098a4-9fe0-4f6b-a8d4-2460a99b79d3-kube-api-access-b2dnz\") pod \"authorino-6f6944bcdd-hb24p\" (UID: \"891098a4-9fe0-4f6b-a8d4-2460a99b79d3\") " pod="kuadrant-system/authorino-6f6944bcdd-hb24p" Oct 30 19:57:16.157636 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.157583 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:16.189297 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.189267 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-tls-cert\") pod \"authorino-6889cbd8c4-49xvf\" (UID: \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\") " pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.189397 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.189369 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z59l\" (UniqueName: \"kubernetes.io/projected/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-kube-api-access-7z59l\") pod \"authorino-6889cbd8c4-49xvf\" (UID: \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\") " pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.286503 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.286473 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-49xvf"] Oct 30 19:57:16.286692 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:57:16.286670 6411 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7z59l tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6889cbd8c4-49xvf" podUID="a9ce8d5e-2f7c-4452-bc4e-b08989eceec9" Oct 30 19:57:16.289972 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.289948 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-tls-cert\") pod \"authorino-6889cbd8c4-49xvf\" (UID: \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\") " pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.290039 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.290018 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z59l\" (UniqueName: \"kubernetes.io/projected/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-kube-api-access-7z59l\") pod \"authorino-6889cbd8c4-49xvf\" (UID: \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\") " pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.292495 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.292477 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-tls-cert\") pod \"authorino-6889cbd8c4-49xvf\" (UID: \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\") " pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.319447 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.319418 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z59l\" (UniqueName: \"kubernetes.io/projected/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-kube-api-access-7z59l\") pod \"authorino-6889cbd8c4-49xvf\" (UID: \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\") " pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.325335 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.325315 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-96bcf7f64-plnjb"] Oct 30 19:57:16.328520 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.328504 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:57:16.349666 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.349639 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-96bcf7f64-plnjb"] Oct 30 19:57:16.390886 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.390863 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec08818f-6463-481d-881a-f3aa9fb082ba-tls-cert\") pod \"authorino-96bcf7f64-plnjb\" (UID: \"ec08818f-6463-481d-881a-f3aa9fb082ba\") " pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:57:16.391010 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.390979 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cttrt\" (UniqueName: \"kubernetes.io/projected/ec08818f-6463-481d-881a-f3aa9fb082ba-kube-api-access-cttrt\") pod \"authorino-96bcf7f64-plnjb\" (UID: \"ec08818f-6463-481d-881a-f3aa9fb082ba\") " pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:57:16.492045 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.491971 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec08818f-6463-481d-881a-f3aa9fb082ba-tls-cert\") pod \"authorino-96bcf7f64-plnjb\" (UID: \"ec08818f-6463-481d-881a-f3aa9fb082ba\") " pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:57:16.492045 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.492018 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cttrt\" (UniqueName: \"kubernetes.io/projected/ec08818f-6463-481d-881a-f3aa9fb082ba-kube-api-access-cttrt\") pod \"authorino-96bcf7f64-plnjb\" (UID: \"ec08818f-6463-481d-881a-f3aa9fb082ba\") " pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:57:16.494422 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.494398 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec08818f-6463-481d-881a-f3aa9fb082ba-tls-cert\") pod \"authorino-96bcf7f64-plnjb\" (UID: \"ec08818f-6463-481d-881a-f3aa9fb082ba\") " pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:57:16.517495 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.517472 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cttrt\" (UniqueName: \"kubernetes.io/projected/ec08818f-6463-481d-881a-f3aa9fb082ba-kube-api-access-cttrt\") pod \"authorino-96bcf7f64-plnjb\" (UID: \"ec08818f-6463-481d-881a-f3aa9fb082ba\") " pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:57:16.640163 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.640135 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:57:16.765802 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.765733 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.765958 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.765733 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-hb24p" Oct 30 19:57:16.772793 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.772772 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:16.775337 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.775319 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-hb24p" Oct 30 19:57:16.895888 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.895859 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/891098a4-9fe0-4f6b-a8d4-2460a99b79d3-kube-api-access-b2dnz\" (UniqueName: \"kubernetes.io/projected/891098a4-9fe0-4f6b-a8d4-2460a99b79d3-kube-api-access-b2dnz\") pod \"891098a4-9fe0-4f6b-a8d4-2460a99b79d3\" (UID: \"891098a4-9fe0-4f6b-a8d4-2460a99b79d3\") " Oct 30 19:57:16.895888 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.895914 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-tls-cert\" (UniqueName: \"kubernetes.io/secret/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-tls-cert\") pod \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\" (UID: \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\") " Oct 30 19:57:16.896164 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.895949 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-kube-api-access-7z59l\" (UniqueName: \"kubernetes.io/projected/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-kube-api-access-7z59l\") pod \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\" (UID: \"a9ce8d5e-2f7c-4452-bc4e-b08989eceec9\") " Oct 30 19:57:16.898204 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.898179 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/891098a4-9fe0-4f6b-a8d4-2460a99b79d3-kube-api-access-b2dnz" pod "891098a4-9fe0-4f6b-a8d4-2460a99b79d3" (UID: "891098a4-9fe0-4f6b-a8d4-2460a99b79d3"). InnerVolumeSpecName "kube-api-access-b2dnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:57:16.898397 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.898362 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-kube-api-access-7z59l" pod "a9ce8d5e-2f7c-4452-bc4e-b08989eceec9" (UID: "a9ce8d5e-2f7c-4452-bc4e-b08989eceec9"). InnerVolumeSpecName "kube-api-access-7z59l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:57:16.898507 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.898401 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-tls-cert" pod "a9ce8d5e-2f7c-4452-bc4e-b08989eceec9" (UID: "a9ce8d5e-2f7c-4452-bc4e-b08989eceec9"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:57:16.934571 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.934537 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-96bcf7f64-plnjb"] Oct 30 19:57:16.937608 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:57:16.937585 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec08818f_6463_481d_881a_f3aa9fb082ba.slice/crio-9df9beaab4f224485da4d37d30e61b0027d71ea8c748129e0a2471661b337708 WatchSource:0}: Error finding container 9df9beaab4f224485da4d37d30e61b0027d71ea8c748129e0a2471661b337708: Status 404 returned error can't find the container with id 9df9beaab4f224485da4d37d30e61b0027d71ea8c748129e0a2471661b337708 Oct 30 19:57:16.938961 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.938942 6411 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:57:16.997012 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.996988 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2dnz\" (UniqueName: \"kubernetes.io/projected/891098a4-9fe0-4f6b-a8d4-2460a99b79d3-kube-api-access-b2dnz\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:57:16.997012 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.997008 6411 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-tls-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:57:16.997137 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:16.997017 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z59l\" (UniqueName: \"kubernetes.io/projected/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9-kube-api-access-7z59l\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:57:17.771048 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.771025 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-hb24p" Oct 30 19:57:17.771048 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.771026 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-96bcf7f64-plnjb" event={"ID":"ec08818f-6463-481d-881a-f3aa9fb082ba","Type":"ContainerStarted","Data":"d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864"} Oct 30 19:57:17.771449 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.771074 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-96bcf7f64-plnjb" event={"ID":"ec08818f-6463-481d-881a-f3aa9fb082ba","Type":"ContainerStarted","Data":"9df9beaab4f224485da4d37d30e61b0027d71ea8c748129e0a2471661b337708"} Oct 30 19:57:17.771449 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.771175 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-49xvf" Oct 30 19:57:17.831658 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.831608 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-96bcf7f64-plnjb" podStartSLOduration=1.395126755 podStartE2EDuration="1.831596084s" totalImagesPullingTime="436.469329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:57:17.830505329 +0000 UTC m=+554.230677337" watchObservedRunningTime="2026-10-30 19:57:17.831596084 +0000 UTC m=+554.231768090" Oct 30 19:57:17.899791 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.899758 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-hb24p"] Oct 30 19:57:17.903085 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.903057 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-hb24p"] Oct 30 19:57:17.967464 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.967435 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-49xvf"] Oct 30 19:57:17.975059 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:17.975035 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-49xvf"] Oct 30 19:57:19.730606 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:19.730575 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="891098a4-9fe0-4f6b-a8d4-2460a99b79d3" path="/var/lib/kubelet/pods/891098a4-9fe0-4f6b-a8d4-2460a99b79d3/volumes" Oct 30 19:57:19.731024 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:19.730810 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9ce8d5e-2f7c-4452-bc4e-b08989eceec9" path="/var/lib/kubelet/pods/a9ce8d5e-2f7c-4452-bc4e-b08989eceec9/volumes" Oct 30 19:57:25.388859 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:25.388826 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:25.580468 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:25.580434 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:25.745868 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:25.745797 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:25.895881 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:25.895857 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:27.315480 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:27.315448 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:29.195796 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:29.195758 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:29.383688 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:29.383652 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:33.108770 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:33.108737 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:33.247649 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:57:33.247609 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:57:35.091026 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:57:35.090965 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:58:01.808571 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:01.808538 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:02.087525 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:02.087435 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:02.399888 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:02.399862 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:02.765346 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:02.765262 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:03.016588 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.016501 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:03.313872 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.313797 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:03.618653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.618557 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:03.686527 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.686500 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-mcct8"] Oct 30 19:58:03.689752 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.689732 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.725144 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.725123 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 19:58:03.725259 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.725208 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Oct 30 19:58:03.725259 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.725221 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7wqnw\"" Oct 30 19:58:03.725992 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.725978 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 19:58:03.726053 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.725978 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 30 19:58:03.738804 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.738784 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-mcct8"] Oct 30 19:58:03.771967 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.771807 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwhdr\" (UniqueName: \"kubernetes.io/projected/68d76a19-a383-41d9-8321-eb7f74680af0-kube-api-access-pwhdr\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.773953 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.772476 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d76a19-a383-41d9-8321-eb7f74680af0-openshift-service-ca\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.773953 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.772511 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-metrics-certs\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.773953 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.773001 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-maas-api-tls\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.838942 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.838914 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:03.875682 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.875598 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwhdr\" (UniqueName: \"kubernetes.io/projected/68d76a19-a383-41d9-8321-eb7f74680af0-kube-api-access-pwhdr\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.875682 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.875662 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d76a19-a383-41d9-8321-eb7f74680af0-openshift-service-ca\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.875872 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.875687 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-metrics-certs\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.875872 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.875726 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-maas-api-tls\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.876279 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.876248 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d76a19-a383-41d9-8321-eb7f74680af0-openshift-service-ca\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.878211 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.878188 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-metrics-certs\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.878336 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.878318 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-maas-api-tls\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:03.901162 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:03.901136 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwhdr\" (UniqueName: \"kubernetes.io/projected/68d76a19-a383-41d9-8321-eb7f74680af0-kube-api-access-pwhdr\") pod \"maas-api-5c48877c75-mcct8\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:04.001384 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:04.001350 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:04.151746 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:04.151697 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:04.361061 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:04.361038 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-mcct8"] Oct 30 19:58:04.363241 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:58:04.363215 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d76a19_a383_41d9_8321_eb7f74680af0.slice/crio-8fce221f6907ee4bbfb3526dbda2071b5da8a1b583ce334fbc6741412ba2a000 WatchSource:0}: Error finding container 8fce221f6907ee4bbfb3526dbda2071b5da8a1b583ce334fbc6741412ba2a000: Status 404 returned error can't find the container with id 8fce221f6907ee4bbfb3526dbda2071b5da8a1b583ce334fbc6741412ba2a000 Oct 30 19:58:04.943883 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:04.943846 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" event={"ID":"68d76a19-a383-41d9-8321-eb7f74680af0","Type":"ContainerStarted","Data":"8fce221f6907ee4bbfb3526dbda2071b5da8a1b583ce334fbc6741412ba2a000"} Oct 30 19:58:05.096240 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:58:05.096201 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:58:05.641202 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:05.641137 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:06.954593 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:06.954508 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" event={"ID":"68d76a19-a383-41d9-8321-eb7f74680af0","Type":"ContainerStarted","Data":"e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419"} Oct 30 19:58:06.954996 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:06.954640 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:07.003572 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:07.003520 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" podStartSLOduration=1.717197719 podStartE2EDuration="4.00350581s" totalImagesPullingTime="2.286308091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:07.002877756 +0000 UTC m=+603.403049765" watchObservedRunningTime="2026-10-30 19:58:07.00350581 +0000 UTC m=+603.403677817" Oct 30 19:58:08.002669 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:08.002635 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:08.188761 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:08.188707 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:08.427808 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:08.427776 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:09.568031 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:09.567995 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:12.968067 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:12.968042 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 19:58:24.706818 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:24.706776 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:29.105750 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:29.105710 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:35.092069 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:58:35.092026 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:58:48.446062 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:48.446023 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:58:48.746675 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:58:48.746596 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:02.867241 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.867194 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6"] Oct 30 19:59:02.870675 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.870656 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:02.888692 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.888673 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Oct 30 19:59:02.900760 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.900736 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Oct 30 19:59:02.900909 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.900878 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Oct 30 19:59:02.900958 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.900923 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xlfkq\"" Oct 30 19:59:02.901989 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.901963 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6"] Oct 30 19:59:02.919736 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.919710 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:02.930842 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.930814 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:02.930968 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.930851 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9915d4-4851-42e5-bd0f-867a621026ee-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:02.930968 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.930890 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrglj\" (UniqueName: \"kubernetes.io/projected/db9915d4-4851-42e5-bd0f-867a621026ee-kube-api-access-jrglj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:02.931090 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.930989 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:02.931090 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.931037 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:02.931172 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:02.931116 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.031706 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.031671 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.031844 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.031713 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.031844 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.031761 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.031844 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.031791 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9915d4-4851-42e5-bd0f-867a621026ee-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.031844 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.031832 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrglj\" (UniqueName: \"kubernetes.io/projected/db9915d4-4851-42e5-bd0f-867a621026ee-kube-api-access-jrglj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.032041 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.031862 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.032245 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.032220 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.032330 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.032249 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.032330 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.032296 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.034220 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.034198 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db9915d4-4851-42e5-bd0f-867a621026ee-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.034318 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.034288 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9915d4-4851-42e5-bd0f-867a621026ee-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.058552 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.058526 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrglj\" (UniqueName: \"kubernetes.io/projected/db9915d4-4851-42e5-bd0f-867a621026ee-kube-api-access-jrglj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6\" (UID: \"db9915d4-4851-42e5-bd0f-867a621026ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.185356 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.185327 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:03.483453 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:03.483380 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6"] Oct 30 19:59:03.486336 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:03.486303 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9915d4_4851_42e5_bd0f_867a621026ee.slice/crio-ce4c16f56830cf22fcc9f827633282743794909bda057d459c099b5b95c90818 WatchSource:0}: Error finding container ce4c16f56830cf22fcc9f827633282743794909bda057d459c099b5b95c90818: Status 404 returned error can't find the container with id ce4c16f56830cf22fcc9f827633282743794909bda057d459c099b5b95c90818 Oct 30 19:59:04.190940 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.190195 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" event={"ID":"db9915d4-4851-42e5-bd0f-867a621026ee","Type":"ContainerStarted","Data":"ce4c16f56830cf22fcc9f827633282743794909bda057d459c099b5b95c90818"} Oct 30 19:59:04.884752 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.884721 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz"] Oct 30 19:59:04.888612 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.888591 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:04.916024 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.915998 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Oct 30 19:59:04.929654 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.929620 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz"] Oct 30 19:59:04.952881 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.952853 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:04.953073 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.952926 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:04.953073 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.953047 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:04.953177 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.953087 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:04.953177 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.953117 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:04.953244 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.953174 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bpc4\" (UniqueName: \"kubernetes.io/projected/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-kube-api-access-6bpc4\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:04.973740 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:04.973699 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:05.054174 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054140 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.054331 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054196 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.054331 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054302 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.054430 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054338 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.054430 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054366 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bpc4\" (UniqueName: \"kubernetes.io/projected/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-kube-api-access-6bpc4\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.054531 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054433 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.054617 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054587 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.054887 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054854 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.054887 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.054872 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.056705 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.056686 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.057259 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.057242 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.083856 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.083825 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bpc4\" (UniqueName: \"kubernetes.io/projected/1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb-kube-api-access-6bpc4\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz\" (UID: \"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.095097 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:59:05.095066 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:59:05.195620 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.195532 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" event={"ID":"db9915d4-4851-42e5-bd0f-867a621026ee","Type":"ContainerStarted","Data":"b78294d151d4cae9ca6e85e7f191c37b87dcc8a4249e5a2a53ed9052efebd11b"} Oct 30 19:59:05.204994 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.204964 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:05.247418 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.247364 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" podStartSLOduration=2.092667106 podStartE2EDuration="3.247348525s" totalImagesPullingTime="1.154681419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:05.245630189 +0000 UTC m=+661.645802196" watchObservedRunningTime="2026-10-30 19:59:05.247348525 +0000 UTC m=+661.647520532" Oct 30 19:59:05.502331 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:05.502243 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz"] Oct 30 19:59:05.505767 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:05.505736 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5f6daf_7e26_4776_ae87_aadb1a4ffcbb.slice/crio-0e21e9003331ae2b669825079da37c4c14b859d6b4618c036dd32284ea7afd80 WatchSource:0}: Error finding container 0e21e9003331ae2b669825079da37c4c14b859d6b4618c036dd32284ea7afd80: Status 404 returned error can't find the container with id 0e21e9003331ae2b669825079da37c4c14b859d6b4618c036dd32284ea7afd80 Oct 30 19:59:06.185618 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:06.185578 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:06.201669 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:06.201635 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" event={"ID":"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb","Type":"ContainerStarted","Data":"4cee6caa539edf18ce3954a0000c0d75449ac0dce1252111f151e15e38926866"} Oct 30 19:59:06.202211 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:06.201687 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" event={"ID":"1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb","Type":"ContainerStarted","Data":"0e21e9003331ae2b669825079da37c4c14b859d6b4618c036dd32284ea7afd80"} Oct 30 19:59:06.205313 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:06.205279 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:06.248744 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:06.248680 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" podStartSLOduration=2.063617265 podStartE2EDuration="2.248661555s" totalImagesPullingTime="185.04429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:06.247499173 +0000 UTC m=+662.647671162" watchObservedRunningTime="2026-10-30 19:59:06.248661555 +0000 UTC m=+662.648833569" Oct 30 19:59:07.451359 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:07.451323 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:09.263918 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.263665 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx"] Oct 30 19:59:09.269918 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.268472 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.285937 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.285890 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wbdhc\"" Oct 30 19:59:09.298684 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.298657 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 19:59:09.303868 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.303838 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx"] Oct 30 19:59:09.396696 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.396658 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.396934 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.396722 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.396934 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.396803 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.396934 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.396834 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.396934 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.396852 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz9zj\" (UniqueName: \"kubernetes.io/projected/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-kube-api-access-hz9zj\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.396934 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.396874 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.397239 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.397049 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.397239 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.397121 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.397239 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.397170 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.498661 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.498618 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.498661 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.498664 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.498952 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.498686 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz9zj\" (UniqueName: \"kubernetes.io/projected/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-kube-api-access-hz9zj\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.498952 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.498716 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.498952 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.498754 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.498952 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.498805 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.499156 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.498946 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.499156 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.499016 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.499156 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.499050 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.499156 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.499131 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.499328 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.499226 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.499373 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.499340 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.499437 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.499420 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.499762 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.499735 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.501521 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.501502 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.501645 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.501622 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.528706 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.528632 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz9zj\" (UniqueName: \"kubernetes.io/projected/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-kube-api-access-hz9zj\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.528706 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.528633 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3130e2e4-4b22-4b49-9a5c-542fc9b3ada9-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-dh2bx\" (UID: \"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.531618 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.531585 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:09.586487 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.586452 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:09.769825 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.769792 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:09.904817 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:09.904773 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx"] Oct 30 19:59:09.907657 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:09.907627 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3130e2e4_4b22_4b49_9a5c_542fc9b3ada9.slice/crio-00d0e7b15cbc9423e35c60771db839cc373d174083ade078f548afe45bf2a485 WatchSource:0}: Error finding container 00d0e7b15cbc9423e35c60771db839cc373d174083ade078f548afe45bf2a485: Status 404 returned error can't find the container with id 00d0e7b15cbc9423e35c60771db839cc373d174083ade078f548afe45bf2a485 Oct 30 19:59:10.219853 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:10.219767 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" event={"ID":"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9","Type":"ContainerStarted","Data":"00d0e7b15cbc9423e35c60771db839cc373d174083ade078f548afe45bf2a485"} Oct 30 19:59:12.166983 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.166948 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2"] Oct 30 19:59:12.174056 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.174024 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.192705 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.192676 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Oct 30 19:59:12.208934 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.205609 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2"] Oct 30 19:59:12.223567 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.223529 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.223741 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.223593 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.223741 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.223657 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2182470d-4038-4ea2-9aaf-930c7653e2dd-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.223741 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.223686 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.223741 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.223706 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg5cb\" (UniqueName: \"kubernetes.io/projected/2182470d-4038-4ea2-9aaf-930c7653e2dd-kube-api-access-gg5cb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.223741 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.223740 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.227664 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.227630 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:12.324806 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.324754 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.325033 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.324844 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.325033 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.324909 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.325033 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.324942 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2182470d-4038-4ea2-9aaf-930c7653e2dd-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.325033 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.324967 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.325033 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.324988 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg5cb\" (UniqueName: \"kubernetes.io/projected/2182470d-4038-4ea2-9aaf-930c7653e2dd-kube-api-access-gg5cb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.325636 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.325309 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.325636 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.325558 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.325636 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.325577 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.327974 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.327951 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2182470d-4038-4ea2-9aaf-930c7653e2dd-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.330359 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.330331 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2182470d-4038-4ea2-9aaf-930c7653e2dd-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.354603 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.354576 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg5cb\" (UniqueName: \"kubernetes.io/projected/2182470d-4038-4ea2-9aaf-930c7653e2dd-kube-api-access-gg5cb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-j9kd2\" (UID: \"2182470d-4038-4ea2-9aaf-930c7653e2dd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.491161 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.491054 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:12.807564 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:12.807480 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2"] Oct 30 19:59:12.811970 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:12.811934 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2182470d_4038_4ea2_9aaf_930c7653e2dd.slice/crio-c74951614c31b4bfd30d90957155f9022042d30fcc22550e324d21252b4e9f86 WatchSource:0}: Error finding container c74951614c31b4bfd30d90957155f9022042d30fcc22550e324d21252b4e9f86: Status 404 returned error can't find the container with id c74951614c31b4bfd30d90957155f9022042d30fcc22550e324d21252b4e9f86 Oct 30 19:59:13.045407 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.045366 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:13.045539 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.045439 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:13.045539 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.045466 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:13.186349 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.186306 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:13.205983 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.205674 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:13.238312 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.238267 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" event={"ID":"3130e2e4-4b22-4b49-9a5c-542fc9b3ada9","Type":"ContainerStarted","Data":"ed0b684c640e49d609a42ced0565b50036e82e71ab8d54a1b45dabde71fe73a7"} Oct 30 19:59:13.243441 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.243404 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" event={"ID":"2182470d-4038-4ea2-9aaf-930c7653e2dd","Type":"ContainerStarted","Data":"1ecec64a529e7bc588155ac7aac0bc7d5fdecbd9f9adab27baf8a68d43e99941"} Oct 30 19:59:13.243600 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.243465 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" event={"ID":"2182470d-4038-4ea2-9aaf-930c7653e2dd","Type":"ContainerStarted","Data":"c74951614c31b4bfd30d90957155f9022042d30fcc22550e324d21252b4e9f86"} Oct 30 19:59:13.261763 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.261733 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6" Oct 30 19:59:13.300699 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.300195 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" podStartSLOduration=1.0707463640000001 podStartE2EDuration="1.300175601s" totalImagesPullingTime="229.429237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:13.299472266 +0000 UTC m=+669.699644273" watchObservedRunningTime="2026-10-30 19:59:13.300175601 +0000 UTC m=+669.700347612" Oct 30 19:59:13.352112 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.352005 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podStartSLOduration=1.217492638 podStartE2EDuration="4.351989094s" totalImagesPullingTime="3.134496456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:13.348253402 +0000 UTC m=+669.748425410" watchObservedRunningTime="2026-10-30 19:59:13.351989094 +0000 UTC m=+669.752161095" Oct 30 19:59:13.491712 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.491673 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:13.589922 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:13.588006 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:14.335635 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:14.335597 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:14.608485 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:14.607035 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Oct 30 19:59:14.608485 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:14.607105 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Oct 30 19:59:15.206187 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:15.206144 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:15.223486 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:15.223460 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:15.270491 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:15.270461 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz" Oct 30 19:59:15.601981 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:15.593301 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Oct 30 19:59:15.601981 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:15.593364 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Oct 30 19:59:16.599924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:16.597283 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Oct 30 19:59:16.599924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:16.597346 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Oct 30 19:59:17.619119 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:17.617289 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Oct 30 19:59:17.619119 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:17.617383 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Oct 30 19:59:17.619119 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:17.618055 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:18.593319 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:18.593284 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Oct 30 19:59:18.593634 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:18.593587 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Oct 30 19:59:19.587882 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:19.587839 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Oct 30 19:59:19.588406 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:19.587947 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Oct 30 19:59:21.629117 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:21.629070 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:59:21.629566 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:21.629150 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:59:22.491767 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:22.491720 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:22.504477 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:22.504457 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:22.629614 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:22.629579 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:59:22.630018 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:22.629638 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:59:23.321673 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:23.321641 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-j9kd2" Oct 30 19:59:23.672025 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:23.671991 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:59:23.672424 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:23.672036 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:59:24.024183 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:24.024105 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:24.024519 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:24.024498 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:24.025204 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:24.025184 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" Oct 30 19:59:25.078857 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.078824 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68db7997f9-9r6gr"] Oct 30 19:59:25.082970 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.082945 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68db7997f9-9r6gr" Oct 30 19:59:25.108431 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.108402 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68db7997f9-9r6gr"] Oct 30 19:59:25.175808 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.175768 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4ct7\" (UniqueName: \"kubernetes.io/projected/92343fb7-b5f2-42c6-a516-aaa881456c5f-kube-api-access-d4ct7\") pod \"authorino-68db7997f9-9r6gr\" (UID: \"92343fb7-b5f2-42c6-a516-aaa881456c5f\") " pod="kuadrant-system/authorino-68db7997f9-9r6gr" Oct 30 19:59:25.175989 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.175835 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/92343fb7-b5f2-42c6-a516-aaa881456c5f-tls-cert\") pod \"authorino-68db7997f9-9r6gr\" (UID: \"92343fb7-b5f2-42c6-a516-aaa881456c5f\") " pod="kuadrant-system/authorino-68db7997f9-9r6gr" Oct 30 19:59:25.276909 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.276868 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4ct7\" (UniqueName: \"kubernetes.io/projected/92343fb7-b5f2-42c6-a516-aaa881456c5f-kube-api-access-d4ct7\") pod \"authorino-68db7997f9-9r6gr\" (UID: \"92343fb7-b5f2-42c6-a516-aaa881456c5f\") " pod="kuadrant-system/authorino-68db7997f9-9r6gr" Oct 30 19:59:25.277067 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.276935 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/92343fb7-b5f2-42c6-a516-aaa881456c5f-tls-cert\") pod \"authorino-68db7997f9-9r6gr\" (UID: \"92343fb7-b5f2-42c6-a516-aaa881456c5f\") " pod="kuadrant-system/authorino-68db7997f9-9r6gr" Oct 30 19:59:25.280075 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.280046 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/92343fb7-b5f2-42c6-a516-aaa881456c5f-tls-cert\") pod \"authorino-68db7997f9-9r6gr\" (UID: \"92343fb7-b5f2-42c6-a516-aaa881456c5f\") " pod="kuadrant-system/authorino-68db7997f9-9r6gr" Oct 30 19:59:25.302330 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.302302 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4ct7\" (UniqueName: \"kubernetes.io/projected/92343fb7-b5f2-42c6-a516-aaa881456c5f-kube-api-access-d4ct7\") pod \"authorino-68db7997f9-9r6gr\" (UID: \"92343fb7-b5f2-42c6-a516-aaa881456c5f\") " pod="kuadrant-system/authorino-68db7997f9-9r6gr" Oct 30 19:59:25.399050 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.399017 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68db7997f9-9r6gr" Oct 30 19:59:25.701436 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:25.701362 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68db7997f9-9r6gr"] Oct 30 19:59:25.704819 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:25.704789 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92343fb7_b5f2_42c6_a516_aaa881456c5f.slice/crio-8534eb93ecda3af9b6eca2667ad2bd9947ed5338fbfc1276eb2030313fd02e74 WatchSource:0}: Error finding container 8534eb93ecda3af9b6eca2667ad2bd9947ed5338fbfc1276eb2030313fd02e74: Status 404 returned error can't find the container with id 8534eb93ecda3af9b6eca2667ad2bd9947ed5338fbfc1276eb2030313fd02e74 Oct 30 19:59:26.320880 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.320846 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68db7997f9-9r6gr" event={"ID":"92343fb7-b5f2-42c6-a516-aaa881456c5f","Type":"ContainerStarted","Data":"508e94cd2979526b35d28726b44bdb9da69c880861ac514819ff7dd2dbac6c8f"} Oct 30 19:59:26.321220 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.320915 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68db7997f9-9r6gr" event={"ID":"92343fb7-b5f2-42c6-a516-aaa881456c5f","Type":"ContainerStarted","Data":"8534eb93ecda3af9b6eca2667ad2bd9947ed5338fbfc1276eb2030313fd02e74"} Oct 30 19:59:26.365850 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.365746 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-68db7997f9-9r6gr" podStartSLOduration=0.834749774 podStartE2EDuration="1.365729727s" totalImagesPullingTime="530.979953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:26.365567474 +0000 UTC m=+682.765739491" watchObservedRunningTime="2026-10-30 19:59:26.365729727 +0000 UTC m=+682.765901735" Oct 30 19:59:26.393224 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.393192 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-96bcf7f64-plnjb"] Oct 30 19:59:26.393449 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.393423 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-96bcf7f64-plnjb" podUID="ec08818f-6463-481d-881a-f3aa9fb082ba" containerName="authorino" containerID="cri-o://d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864" gracePeriod=30 Oct 30 19:59:26.646704 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.646673 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:59:26.795312 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.795230 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec08818f-6463-481d-881a-f3aa9fb082ba-kube-api-access-cttrt\" (UniqueName: \"kubernetes.io/projected/ec08818f-6463-481d-881a-f3aa9fb082ba-kube-api-access-cttrt\") pod \"ec08818f-6463-481d-881a-f3aa9fb082ba\" (UID: \"ec08818f-6463-481d-881a-f3aa9fb082ba\") " Oct 30 19:59:26.795481 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.795365 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec08818f-6463-481d-881a-f3aa9fb082ba-tls-cert\" (UniqueName: \"kubernetes.io/secret/ec08818f-6463-481d-881a-f3aa9fb082ba-tls-cert\") pod \"ec08818f-6463-481d-881a-f3aa9fb082ba\" (UID: \"ec08818f-6463-481d-881a-f3aa9fb082ba\") " Oct 30 19:59:26.797558 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.797515 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec08818f-6463-481d-881a-f3aa9fb082ba-kube-api-access-cttrt" pod "ec08818f-6463-481d-881a-f3aa9fb082ba" (UID: "ec08818f-6463-481d-881a-f3aa9fb082ba"). InnerVolumeSpecName "kube-api-access-cttrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:59:26.806778 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.806747 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec08818f-6463-481d-881a-f3aa9fb082ba-tls-cert" pod "ec08818f-6463-481d-881a-f3aa9fb082ba" (UID: "ec08818f-6463-481d-881a-f3aa9fb082ba"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:59:26.896418 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.896384 6411 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec08818f-6463-481d-881a-f3aa9fb082ba-tls-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:59:26.896418 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:26.896413 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cttrt\" (UniqueName: \"kubernetes.io/projected/ec08818f-6463-481d-881a-f3aa9fb082ba-kube-api-access-cttrt\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:59:27.029833 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.029790 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d6c696b49-pbsvb"] Oct 30 19:59:27.030165 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.030144 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:27.030258 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.030188 6411 cpu_manager.go:414] "containerMap: removing container" podUID="ec08818f-6463-481d-881a-f3aa9fb082ba" containerName="authorino" Oct 30 19:59:27.030258 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.030202 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec08818f-6463-481d-881a-f3aa9fb082ba" containerName="authorino" Oct 30 19:59:27.030340 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.030294 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec08818f-6463-481d-881a-f3aa9fb082ba" containerName="authorino" Oct 30 19:59:27.033417 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.033402 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:27.051181 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.051109 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Oct 30 19:59:27.062511 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.062459 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-m9pls\"" Oct 30 19:59:27.062658 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.062550 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\"" Oct 30 19:59:27.065201 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.065174 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d6c696b49-pbsvb"] Oct 30 19:59:27.199653 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.199618 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d6a818-a397-48b9-a7f0-37ca0e2fef27-metrics-certs\") pod \"ai-gateway-controller-7d6c696b49-pbsvb\" (UID: \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\") " pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:27.199846 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.199736 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6qvk\" (UniqueName: \"kubernetes.io/projected/13d6a818-a397-48b9-a7f0-37ca0e2fef27-kube-api-access-f6qvk\") pod \"ai-gateway-controller-7d6c696b49-pbsvb\" (UID: \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\") " pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:27.227939 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.227877 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:27.301098 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.301070 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d6a818-a397-48b9-a7f0-37ca0e2fef27-metrics-certs\") pod \"ai-gateway-controller-7d6c696b49-pbsvb\" (UID: \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\") " pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:27.301250 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.301137 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6qvk\" (UniqueName: \"kubernetes.io/projected/13d6a818-a397-48b9-a7f0-37ca0e2fef27-kube-api-access-f6qvk\") pod \"ai-gateway-controller-7d6c696b49-pbsvb\" (UID: \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\") " pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:27.303811 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.303783 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d6a818-a397-48b9-a7f0-37ca0e2fef27-metrics-certs\") pod \"ai-gateway-controller-7d6c696b49-pbsvb\" (UID: \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\") " pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:27.327627 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.327591 6411 generic.go:356] "Generic (PLEG): container finished" podID="ec08818f-6463-481d-881a-f3aa9fb082ba" containerID="d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864" exitCode=0 Oct 30 19:59:27.328075 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.327648 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-96bcf7f64-plnjb" Oct 30 19:59:27.328075 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.327683 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-96bcf7f64-plnjb" event={"ID":"ec08818f-6463-481d-881a-f3aa9fb082ba","Type":"ContainerDied","Data":"d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864"} Oct 30 19:59:27.328075 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.327748 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-96bcf7f64-plnjb" event={"ID":"ec08818f-6463-481d-881a-f3aa9fb082ba","Type":"ContainerDied","Data":"9df9beaab4f224485da4d37d30e61b0027d71ea8c748129e0a2471661b337708"} Oct 30 19:59:27.328075 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.327778 6411 scope.go:122] "RemoveContainer" containerID="d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864" Oct 30 19:59:27.330664 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.330644 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6qvk\" (UniqueName: \"kubernetes.io/projected/13d6a818-a397-48b9-a7f0-37ca0e2fef27-kube-api-access-f6qvk\") pod \"ai-gateway-controller-7d6c696b49-pbsvb\" (UID: \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\") " pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:27.344468 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.344447 6411 scope.go:122] "RemoveContainer" containerID="d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864" Oct 30 19:59:27.344785 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:59:27.344766 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864\": container with ID starting with d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864 not found: ID does not exist" containerID="d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864" Oct 30 19:59:27.344849 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.344794 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864"} err="failed to get container status \"d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864\": rpc error: code = NotFound desc = could not find container \"d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864\": container with ID starting with d6e24f64d62320a3834c66eeeb0cbbf92a3cb3f60487d624b074bffda0559864 not found: ID does not exist" Oct 30 19:59:27.347982 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.347965 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:27.382144 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.382115 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-96bcf7f64-plnjb"] Oct 30 19:59:27.385150 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.385127 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-96bcf7f64-plnjb"] Oct 30 19:59:27.653243 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.653209 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d6c696b49-pbsvb"] Oct 30 19:59:27.657445 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:27.657418 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d6a818_a397_48b9_a7f0_37ca0e2fef27.slice/crio-490dc143fd98757a00613d0c3ca0037b6c5b9b0718d14729dccbec8bdc61c4bf WatchSource:0}: Error finding container 490dc143fd98757a00613d0c3ca0037b6c5b9b0718d14729dccbec8bdc61c4bf: Status 404 returned error can't find the container with id 490dc143fd98757a00613d0c3ca0037b6c5b9b0718d14729dccbec8bdc61c4bf Oct 30 19:59:27.732152 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:27.732122 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec08818f-6463-481d-881a-f3aa9fb082ba" path="/var/lib/kubelet/pods/ec08818f-6463-481d-881a-f3aa9fb082ba/volumes" Oct 30 19:59:28.338338 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:28.337454 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" event={"ID":"13d6a818-a397-48b9-a7f0-37ca0e2fef27","Type":"ContainerStarted","Data":"490dc143fd98757a00613d0c3ca0037b6c5b9b0718d14729dccbec8bdc61c4bf"} Oct 30 19:59:31.357166 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:31.357125 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" event={"ID":"13d6a818-a397-48b9-a7f0-37ca0e2fef27","Type":"ContainerStarted","Data":"53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd"} Oct 30 19:59:31.357560 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:31.357266 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:31.404215 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:31.404152 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" podStartSLOduration=2.736155786 podStartE2EDuration="5.404135139s" totalImagesPullingTime="2.667979353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:31.402862707 +0000 UTC m=+687.803034715" watchObservedRunningTime="2026-10-30 19:59:31.404135139 +0000 UTC m=+687.804307147" Oct 30 19:59:32.565050 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.565018 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:32.763494 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.763459 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-xhkln"] Oct 30 19:59:32.766962 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.766943 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:32.768931 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.768849 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:32.796794 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.796768 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pbvh8\"" Oct 30 19:59:32.796940 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.796831 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Oct 30 19:59:32.798065 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.798047 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-xhkln"] Oct 30 19:59:32.856281 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.856195 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtfhj\" (UniqueName: \"kubernetes.io/projected/5ccf9e4e-fb38-41df-8b3d-c7210abcfb74-kube-api-access-gtfhj\") pod \"payload-pre-processing-7b98bb678c-xhkln\" (UID: \"5ccf9e4e-fb38-41df-8b3d-c7210abcfb74\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:32.856281 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.856270 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ccf9e4e-fb38-41df-8b3d-c7210abcfb74-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-xhkln\" (UID: \"5ccf9e4e-fb38-41df-8b3d-c7210abcfb74\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:32.856819 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.856786 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-6bfdff5577-vmghk"] Oct 30 19:59:32.860222 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.860206 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:32.881632 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.881606 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-6bfdff5577-vmghk"] Oct 30 19:59:32.957579 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.957548 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtfhj\" (UniqueName: \"kubernetes.io/projected/5ccf9e4e-fb38-41df-8b3d-c7210abcfb74-kube-api-access-gtfhj\") pod \"payload-pre-processing-7b98bb678c-xhkln\" (UID: \"5ccf9e4e-fb38-41df-8b3d-c7210abcfb74\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:32.957747 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.957602 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flzpz\" (UniqueName: \"kubernetes.io/projected/1f667352-a367-45dd-9b7b-a1b03d2d24a3-kube-api-access-flzpz\") pod \"payload-processing-6bfdff5577-vmghk\" (UID: \"1f667352-a367-45dd-9b7b-a1b03d2d24a3\") " pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:32.957747 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.957645 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ccf9e4e-fb38-41df-8b3d-c7210abcfb74-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-xhkln\" (UID: \"5ccf9e4e-fb38-41df-8b3d-c7210abcfb74\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:32.957888 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.957861 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f667352-a367-45dd-9b7b-a1b03d2d24a3-plugins-config-volume\") pod \"payload-processing-6bfdff5577-vmghk\" (UID: \"1f667352-a367-45dd-9b7b-a1b03d2d24a3\") " pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:32.958369 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.958352 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ccf9e4e-fb38-41df-8b3d-c7210abcfb74-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-xhkln\" (UID: \"5ccf9e4e-fb38-41df-8b3d-c7210abcfb74\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:32.983757 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:32.983724 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtfhj\" (UniqueName: \"kubernetes.io/projected/5ccf9e4e-fb38-41df-8b3d-c7210abcfb74-kube-api-access-gtfhj\") pod \"payload-pre-processing-7b98bb678c-xhkln\" (UID: \"5ccf9e4e-fb38-41df-8b3d-c7210abcfb74\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:33.059116 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:33.059086 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f667352-a367-45dd-9b7b-a1b03d2d24a3-plugins-config-volume\") pod \"payload-processing-6bfdff5577-vmghk\" (UID: \"1f667352-a367-45dd-9b7b-a1b03d2d24a3\") " pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:33.059297 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:33.059140 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flzpz\" (UniqueName: \"kubernetes.io/projected/1f667352-a367-45dd-9b7b-a1b03d2d24a3-kube-api-access-flzpz\") pod \"payload-processing-6bfdff5577-vmghk\" (UID: \"1f667352-a367-45dd-9b7b-a1b03d2d24a3\") " pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:33.059913 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:33.059871 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f667352-a367-45dd-9b7b-a1b03d2d24a3-plugins-config-volume\") pod \"payload-processing-6bfdff5577-vmghk\" (UID: \"1f667352-a367-45dd-9b7b-a1b03d2d24a3\") " pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:33.080637 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:33.080609 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:33.085953 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:33.085926 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flzpz\" (UniqueName: \"kubernetes.io/projected/1f667352-a367-45dd-9b7b-a1b03d2d24a3-kube-api-access-flzpz\") pod \"payload-processing-6bfdff5577-vmghk\" (UID: \"1f667352-a367-45dd-9b7b-a1b03d2d24a3\") " pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:33.176082 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:33.176053 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:33.374113 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:33.374085 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-xhkln"] Oct 30 19:59:33.377183 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:33.377153 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ccf9e4e_fb38_41df_8b3d_c7210abcfb74.slice/crio-0d4ff4a1421b32ca9a8f66e3529ea5b555c6161b0e0941295e340daf91b39705 WatchSource:0}: Error finding container 0d4ff4a1421b32ca9a8f66e3529ea5b555c6161b0e0941295e340daf91b39705: Status 404 returned error can't find the container with id 0d4ff4a1421b32ca9a8f66e3529ea5b555c6161b0e0941295e340daf91b39705 Oct 30 19:59:33.473562 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:33.473533 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-6bfdff5577-vmghk"] Oct 30 19:59:33.476600 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:33.476577 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f667352_a367_45dd_9b7b_a1b03d2d24a3.slice/crio-13e27edf52774eb115cb407f9641085e4575ed9bd631d40a1ebedee2ff75f1e0 WatchSource:0}: Error finding container 13e27edf52774eb115cb407f9641085e4575ed9bd631d40a1ebedee2ff75f1e0: Status 404 returned error can't find the container with id 13e27edf52774eb115cb407f9641085e4575ed9bd631d40a1ebedee2ff75f1e0 Oct 30 19:59:34.371732 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:34.371686 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" event={"ID":"1f667352-a367-45dd-9b7b-a1b03d2d24a3","Type":"ContainerStarted","Data":"13e27edf52774eb115cb407f9641085e4575ed9bd631d40a1ebedee2ff75f1e0"} Oct 30 19:59:34.373587 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:34.373563 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" event={"ID":"5ccf9e4e-fb38-41df-8b3d-c7210abcfb74","Type":"ContainerStarted","Data":"0d4ff4a1421b32ca9a8f66e3529ea5b555c6161b0e0941295e340daf91b39705"} Oct 30 19:59:35.106972 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:59:35.106929 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 19:59:36.384417 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:36.384377 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" event={"ID":"1f667352-a367-45dd-9b7b-a1b03d2d24a3","Type":"ContainerStarted","Data":"e2912e96db46a5490b796a9448eff036044ac51b36141c9e8d3b0899a05ad994"} Oct 30 19:59:36.384878 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:36.384479 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:36.386049 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:36.386028 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" event={"ID":"5ccf9e4e-fb38-41df-8b3d-c7210abcfb74","Type":"ContainerStarted","Data":"0fde00a63dc4c45d53cba00e3c1785160774f58ef1d49bb255e645c943845053"} Oct 30 19:59:36.386190 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:36.386081 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:36.431046 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:36.431006 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" podStartSLOduration=2.255955344 podStartE2EDuration="4.43099562s" totalImagesPullingTime="2.175040276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:36.429948671 +0000 UTC m=+692.830120680" watchObservedRunningTime="2026-10-30 19:59:36.43099562 +0000 UTC m=+692.831167627" Oct 30 19:59:36.474437 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:36.474395 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" podStartSLOduration=2.305544029 podStartE2EDuration="4.474382659s" totalImagesPullingTime="2.16883863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:36.473208953 +0000 UTC m=+692.873380960" watchObservedRunningTime="2026-10-30 19:59:36.474382659 +0000 UTC m=+692.874554665" Oct 30 19:59:39.365614 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.364839 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv"] Oct 30 19:59:39.377512 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.377486 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.383701 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.383667 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp"] Oct 30 19:59:39.388487 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.388465 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.395252 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.395230 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv"] Oct 30 19:59:39.414995 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.414971 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp"] Oct 30 19:59:39.520222 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520191 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.520352 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520226 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520352 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520255 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520352 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520310 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.520485 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520412 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lsrh\" (UniqueName: \"kubernetes.io/projected/f2139ca9-f068-4350-8d67-00ce2a968911-kube-api-access-4lsrh\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520485 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520462 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520548 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520525 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.520582 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520570 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.520614 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520586 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.520647 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520613 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2139ca9-f068-4350-8d67-00ce2a968911-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520677 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520658 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2139ca9-f068-4350-8d67-00ce2a968911-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520710 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520678 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.520710 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520694 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptvfs\" (UniqueName: \"kubernetes.io/projected/b0a807a8-3f3a-42b0-8fa5-48841828bebc-kube-api-access-ptvfs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.520778 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520761 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520810 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520782 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2139ca9-f068-4350-8d67-00ce2a968911-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520848 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520836 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.520885 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520859 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.520967 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.520887 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.621874 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.621799 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.621874 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.621833 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.621874 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.621851 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622122 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.621960 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622122 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.621996 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622122 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622030 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622122 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622059 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622122 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622086 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lsrh\" (UniqueName: \"kubernetes.io/projected/f2139ca9-f068-4350-8d67-00ce2a968911-kube-api-access-4lsrh\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622130 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622165 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622225 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622231 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622251 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622255 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622309 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2139ca9-f068-4350-8d67-00ce2a968911-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622333 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622370 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622337 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2139ca9-f068-4350-8d67-00ce2a968911-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622372 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622372 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622426 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptvfs\" (UniqueName: \"kubernetes.io/projected/b0a807a8-3f3a-42b0-8fa5-48841828bebc-kube-api-access-ptvfs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622456 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622467 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622516 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.622863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622517 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2139ca9-f068-4350-8d67-00ce2a968911-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.622863 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622833 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.623473 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.622887 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.623473 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.623001 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2139ca9-f068-4350-8d67-00ce2a968911-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.623473 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.623021 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.625042 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.625017 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.625245 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.625224 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2139ca9-f068-4350-8d67-00ce2a968911-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.625338 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.625316 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2139ca9-f068-4350-8d67-00ce2a968911-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.625509 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.625494 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.648110 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.648090 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2139ca9-f068-4350-8d67-00ce2a968911-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.648187 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.648094 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lsrh\" (UniqueName: \"kubernetes.io/projected/f2139ca9-f068-4350-8d67-00ce2a968911-kube-api-access-4lsrh\") pod \"maas-default-gateway-openshift-default-7f5667f86c-smpxp\" (UID: \"f2139ca9-f068-4350-8d67-00ce2a968911\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:39.648323 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.648307 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0a807a8-3f3a-42b0-8fa5-48841828bebc-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.648597 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.648579 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptvfs\" (UniqueName: \"kubernetes.io/projected/b0a807a8-3f3a-42b0-8fa5-48841828bebc-kube-api-access-ptvfs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kplbv\" (UID: \"b0a807a8-3f3a-42b0-8fa5-48841828bebc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.692581 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.692558 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:39.702645 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:39.702625 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:40.020235 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.020209 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv"] Oct 30 19:59:40.022632 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:40.022606 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a807a8_3f3a_42b0_8fa5_48841828bebc.slice/crio-243193b2f64d784df895ebb0e959de022b589fcdf76596ef68b14b33546554a3 WatchSource:0}: Error finding container 243193b2f64d784df895ebb0e959de022b589fcdf76596ef68b14b33546554a3: Status 404 returned error can't find the container with id 243193b2f64d784df895ebb0e959de022b589fcdf76596ef68b14b33546554a3 Oct 30 19:59:40.023273 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.023250 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp"] Oct 30 19:59:40.025077 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.025038 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:40.025142 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.025112 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:40.025178 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.025148 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:40.025646 ip-10-0-142-168 kubenswrapper[6411]: W1030 19:59:40.025529 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2139ca9_f068_4350_8d67_00ce2a968911.slice/crio-81fe00a48b534baa0ff2e4ebaf2e2b42870672666b66d0fa386855f48ada6f90 WatchSource:0}: Error finding container 81fe00a48b534baa0ff2e4ebaf2e2b42870672666b66d0fa386855f48ada6f90: Status 404 returned error can't find the container with id 81fe00a48b534baa0ff2e4ebaf2e2b42870672666b66d0fa386855f48ada6f90 Oct 30 19:59:40.027596 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.027570 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:40.027720 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.027699 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:40.027792 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.027743 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 19:59:40.406377 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.406342 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" event={"ID":"f2139ca9-f068-4350-8d67-00ce2a968911","Type":"ContainerStarted","Data":"9c0bd8515b605f04ac05b773b639bcd1370e88a45ac0552315a29ab351ee60a1"} Oct 30 19:59:40.406765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.406407 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" event={"ID":"f2139ca9-f068-4350-8d67-00ce2a968911","Type":"ContainerStarted","Data":"81fe00a48b534baa0ff2e4ebaf2e2b42870672666b66d0fa386855f48ada6f90"} Oct 30 19:59:40.408480 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.408452 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" event={"ID":"b0a807a8-3f3a-42b0-8fa5-48841828bebc","Type":"ContainerStarted","Data":"1ce3cf1f04b15043ffbbf1758113360429abc7157138e7d25c200683b834d8a5"} Oct 30 19:59:40.408609 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.408500 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" event={"ID":"b0a807a8-3f3a-42b0-8fa5-48841828bebc","Type":"ContainerStarted","Data":"243193b2f64d784df895ebb0e959de022b589fcdf76596ef68b14b33546554a3"} Oct 30 19:59:40.458330 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.458167 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podStartSLOduration=1.45814712 podStartE2EDuration="1.45814712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:40.455977569 +0000 UTC m=+696.856149577" watchObservedRunningTime="2026-10-30 19:59:40.45814712 +0000 UTC m=+696.858319133" Oct 30 19:59:40.694984 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.693010 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:40.716924 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:40.704000 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:41.702700 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:41.702375 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:59:41.702700 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:41.702435 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:59:41.704683 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:41.704655 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-smpxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:59:41.704913 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:41.704870 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podUID="f2139ca9-f068-4350-8d67-00ce2a968911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:59:42.363369 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.363335 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:42.416923 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.416189 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podStartSLOduration=3.41616602 podStartE2EDuration="3.41616602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:40.506667478 +0000 UTC m=+696.906839520" watchObservedRunningTime="2026-10-30 19:59:42.41616602 +0000 UTC m=+698.816338032" Oct 30 19:59:42.670165 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.670127 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6dd98c85c8-5rphq"] Oct 30 19:59:42.674544 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.674516 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:42.703325 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.701354 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6dd98c85c8-5rphq"] Oct 30 19:59:42.703325 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.702612 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:59:42.703325 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.702682 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:59:42.703793 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.703552 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-smpxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:59:42.703793 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.703592 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podUID="f2139ca9-f068-4350-8d67-00ce2a968911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:59:42.851541 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.851488 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7z55\" (UniqueName: \"kubernetes.io/projected/e2c2db81-c0d6-4d72-9fae-24e76a66409c-kube-api-access-h7z55\") pod \"ai-gateway-controller-6dd98c85c8-5rphq\" (UID: \"e2c2db81-c0d6-4d72-9fae-24e76a66409c\") " pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:42.851735 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.851665 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2c2db81-c0d6-4d72-9fae-24e76a66409c-metrics-certs\") pod \"ai-gateway-controller-6dd98c85c8-5rphq\" (UID: \"e2c2db81-c0d6-4d72-9fae-24e76a66409c\") " pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:42.953268 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.953179 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2c2db81-c0d6-4d72-9fae-24e76a66409c-metrics-certs\") pod \"ai-gateway-controller-6dd98c85c8-5rphq\" (UID: \"e2c2db81-c0d6-4d72-9fae-24e76a66409c\") " pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:42.953268 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.953245 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7z55\" (UniqueName: \"kubernetes.io/projected/e2c2db81-c0d6-4d72-9fae-24e76a66409c-kube-api-access-h7z55\") pod \"ai-gateway-controller-6dd98c85c8-5rphq\" (UID: \"e2c2db81-c0d6-4d72-9fae-24e76a66409c\") " pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:42.957321 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.957298 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2c2db81-c0d6-4d72-9fae-24e76a66409c-metrics-certs\") pod \"ai-gateway-controller-6dd98c85c8-5rphq\" (UID: \"e2c2db81-c0d6-4d72-9fae-24e76a66409c\") " pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:42.995024 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:42.994969 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7z55\" (UniqueName: \"kubernetes.io/projected/e2c2db81-c0d6-4d72-9fae-24e76a66409c-kube-api-access-h7z55\") pod \"ai-gateway-controller-6dd98c85c8-5rphq\" (UID: \"e2c2db81-c0d6-4d72-9fae-24e76a66409c\") " pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:43.038765 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:43.038725 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:43.415016 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:43.410182 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6dd98c85c8-5rphq"] Oct 30 19:59:43.450415 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:43.450386 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" event={"ID":"e2c2db81-c0d6-4d72-9fae-24e76a66409c","Type":"ContainerStarted","Data":"96f61016761e64373a86208e3404c66a5ec5d345b065ee36f36a687539f8e3db"} Oct 30 19:59:43.695377 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:43.695281 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:59:43.695623 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:43.695594 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:59:43.703675 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:43.703645 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-smpxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:59:43.704203 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:43.704171 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podUID="f2139ca9-f068-4350-8d67-00ce2a968911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:59:44.456611 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:44.456562 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" event={"ID":"e2c2db81-c0d6-4d72-9fae-24e76a66409c","Type":"ContainerStarted","Data":"cdef9016158fa6b4d836a9b800fee5581b5d2dd9e8e6f00e51e47aafbc4cc879"} Oct 30 19:59:44.456836 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:44.456798 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:44.504922 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:44.504535 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" podStartSLOduration=2.17492015 podStartE2EDuration="2.504514758s" totalImagesPullingTime="329.594608ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:44.504144129 +0000 UTC m=+700.904316137" watchObservedRunningTime="2026-10-30 19:59:44.504514758 +0000 UTC m=+700.904686766" Oct 30 19:59:44.693933 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:44.693284 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:59:44.693933 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:44.693347 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:59:44.709990 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:44.706263 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-smpxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:59:44.709990 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:44.706317 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podUID="f2139ca9-f068-4350-8d67-00ce2a968911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:59:45.694923 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:45.693940 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:59:45.694923 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:45.694005 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:59:45.703920 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:45.703217 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-smpxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:59:45.703920 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:45.703267 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podUID="f2139ca9-f068-4350-8d67-00ce2a968911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:59:46.693966 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:46.693927 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:59:46.694327 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:46.693979 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:59:46.703253 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:46.703229 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-smpxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:59:46.703360 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:46.703271 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podUID="f2139ca9-f068-4350-8d67-00ce2a968911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:59:47.395211 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:47.395182 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-6bfdff5577-vmghk" Oct 30 19:59:47.395409 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:47.395269 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-xhkln" Oct 30 19:59:48.435124 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:48.435082 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:48.435818 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:48.435715 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:48.436654 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:48.436627 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" Oct 30 19:59:48.497615 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:48.497581 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:48.498019 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:48.497999 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:48.499074 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:48.499054 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" Oct 30 19:59:55.463533 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:55.463504 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6dd98c85c8-5rphq" Oct 30 19:59:55.538356 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:55.538324 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d6c696b49-pbsvb"] Oct 30 19:59:55.538573 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:55.538538 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" podUID="13d6a818-a397-48b9-a7f0-37ca0e2fef27" containerName="manager" containerID="cri-o://53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd" gracePeriod=30 Oct 30 19:59:55.789727 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:55.789703 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:55.900054 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:55.900020 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13d6a818-a397-48b9-a7f0-37ca0e2fef27-kube-api-access-f6qvk\" (UniqueName: \"kubernetes.io/projected/13d6a818-a397-48b9-a7f0-37ca0e2fef27-kube-api-access-f6qvk\") pod \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\" (UID: \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\") " Oct 30 19:59:55.900054 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:55.900058 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/13d6a818-a397-48b9-a7f0-37ca0e2fef27-metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d6a818-a397-48b9-a7f0-37ca0e2fef27-metrics-certs\") pod \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\" (UID: \"13d6a818-a397-48b9-a7f0-37ca0e2fef27\") " Oct 30 19:59:55.902379 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:55.902344 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13d6a818-a397-48b9-a7f0-37ca0e2fef27-metrics-certs" pod "13d6a818-a397-48b9-a7f0-37ca0e2fef27" (UID: "13d6a818-a397-48b9-a7f0-37ca0e2fef27"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:59:55.902379 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:55.902366 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d6a818-a397-48b9-a7f0-37ca0e2fef27-kube-api-access-f6qvk" pod "13d6a818-a397-48b9-a7f0-37ca0e2fef27" (UID: "13d6a818-a397-48b9-a7f0-37ca0e2fef27"). InnerVolumeSpecName "kube-api-access-f6qvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:59:56.001615 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.001569 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6qvk\" (UniqueName: \"kubernetes.io/projected/13d6a818-a397-48b9-a7f0-37ca0e2fef27-kube-api-access-f6qvk\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:59:56.001615 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.001608 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d6a818-a397-48b9-a7f0-37ca0e2fef27-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 19:59:56.509305 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.509261 6411 generic.go:356] "Generic (PLEG): container finished" podID="13d6a818-a397-48b9-a7f0-37ca0e2fef27" containerID="53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd" exitCode=0 Oct 30 19:59:56.509690 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.509331 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" Oct 30 19:59:56.509690 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.509339 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" event={"ID":"13d6a818-a397-48b9-a7f0-37ca0e2fef27","Type":"ContainerDied","Data":"53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd"} Oct 30 19:59:56.509690 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.509424 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d6c696b49-pbsvb" event={"ID":"13d6a818-a397-48b9-a7f0-37ca0e2fef27","Type":"ContainerDied","Data":"490dc143fd98757a00613d0c3ca0037b6c5b9b0718d14729dccbec8bdc61c4bf"} Oct 30 19:59:56.509690 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.509455 6411 scope.go:122] "RemoveContainer" containerID="53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd" Oct 30 19:59:56.518057 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.518039 6411 scope.go:122] "RemoveContainer" containerID="53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd" Oct 30 19:59:56.518293 ip-10-0-142-168 kubenswrapper[6411]: E1030 19:59:56.518278 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd\": container with ID starting with 53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd not found: ID does not exist" containerID="53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd" Oct 30 19:59:56.518338 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.518301 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd"} err="failed to get container status \"53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd\": rpc error: code = NotFound desc = could not find container \"53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd\": container with ID starting with 53b5e8bfd0f7c01f1fd9a8ea37b32803b5225224f36bb8056e0c9657bdaf01cd not found: ID does not exist" Oct 30 19:59:56.566111 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.566086 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d6c696b49-pbsvb"] Oct 30 19:59:56.578751 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:56.578715 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d6c696b49-pbsvb"] Oct 30 19:59:57.240546 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:57.240504 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:57.731328 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:57.731295 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13d6a818-a397-48b9-a7f0-37ca0e2fef27" path="/var/lib/kubelet/pods/13d6a818-a397-48b9-a7f0-37ca0e2fef27/volumes" Oct 30 19:59:57.794968 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:57.794942 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:59.502726 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:59.502692 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 19:59:59.714756 ip-10-0-142-168 kubenswrapper[6411]: I1030 19:59:59.714719 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:01.070472 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:01.070427 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:01.273671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:01.273636 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:05.089964 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:00:05.089926 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:00:09.797272 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:09.797237 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:15.466326 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:15.466240 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:23.397143 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.397108 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:23.654810 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.654714 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:23.860573 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.860541 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5"] Oct 30 20:00:23.861003 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.860978 6411 cpu_manager.go:414] "containerMap: removing container" podUID="13d6a818-a397-48b9-a7f0-37ca0e2fef27" containerName="manager" Oct 30 20:00:23.861153 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.861023 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13d6a818-a397-48b9-a7f0-37ca0e2fef27" containerName="manager" Oct 30 20:00:23.861153 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.861130 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="13d6a818-a397-48b9-a7f0-37ca0e2fef27" containerName="manager" Oct 30 20:00:23.864538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.864509 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:23.899382 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.899351 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-f1f016-openshift-default-dockercfg-pgfjl\"" Oct 30 20:00:23.900834 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.900811 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5"] Oct 30 20:00:23.911052 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:23.910971 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:24.060365 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060325 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.060538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060388 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.060538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060476 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.060538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060513 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.060689 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060648 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.060741 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060689 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.060741 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060717 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.060859 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060745 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcvjh\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-kube-api-access-gcvjh\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.060962 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.060865 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.105297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.105258 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:24.161816 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.161726 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.161816 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.161769 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.162196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.161816 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.162196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.161864 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.162196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.161955 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.162196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.161986 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.162196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.162002 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.162196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.162029 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcvjh\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-kube-api-access-gcvjh\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.162196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.162056 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.162772 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.162746 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.164198 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.164172 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.165097 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.165022 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.165237 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.165034 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.165343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.164523 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.174629 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.174597 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.177155 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.177129 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.190308 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.190286 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.190627 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.190608 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcvjh\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-kube-api-access-gcvjh\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.270276 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.270230 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2"] Oct 30 20:00:24.275833 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.275811 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.301257 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.299883 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-62f4a1-openshift-default-dockercfg-hg55k\"" Oct 30 20:00:24.301257 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.301004 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2"] Oct 30 20:00:24.302182 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.302154 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:24.420335 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.420252 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd"] Oct 30 20:00:24.423917 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.423860 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.450141 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.450108 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd"] Oct 30 20:00:24.459985 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.459959 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-5f10d2-openshift-default-dockercfg-mbtgp\"" Oct 30 20:00:24.465401 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465377 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97zn5\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-kube-api-access-97zn5\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.465510 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465410 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.465510 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465443 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.465616 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465592 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.465670 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465651 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.465722 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465708 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.465759 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465742 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.465806 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465770 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71651b4-40f9-4900-aa6d-84328edffa6d-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.465859 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.465844 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.483052 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.483030 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:24.567132 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567099 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.567319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567154 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.567319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567197 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.567319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567228 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.567319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567255 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.567319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567279 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71651b4-40f9-4900-aa6d-84328edffa6d-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.567319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567312 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567365 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567399 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567433 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567458 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khn8r\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-kube-api-access-khn8r\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567492 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567532 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567559 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85e7c420-1e78-42cf-abc6-25366f1e9387-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567596 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97zn5\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-kube-api-access-97zn5\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.567646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567632 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.568153 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567680 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.568153 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.567712 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.568260 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.568203 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.569040 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.569016 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71651b4-40f9-4900-aa6d-84328edffa6d-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.569160 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.569013 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.569294 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.569265 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.569445 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.569421 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.572105 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.572051 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.572556 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.572538 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.593161 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.593125 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:24.598457 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.598432 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.598581 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.598556 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97zn5\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-kube-api-access-97zn5\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:24.668429 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668392 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668586 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668479 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668586 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668554 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668585 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668620 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668651 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khn8r\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-kube-api-access-khn8r\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668816 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668679 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668816 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668716 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668816 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668744 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85e7c420-1e78-42cf-abc6-25366f1e9387-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.668998 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.668937 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.669083 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.669057 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.669213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.669183 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.669329 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.669198 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.669661 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.669641 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85e7c420-1e78-42cf-abc6-25366f1e9387-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.670967 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.670889 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.671089 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.671071 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.696846 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.696816 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khn8r\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-kube-api-access-khn8r\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.697012 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.696816 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.737454 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.737421 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:24.802101 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.801703 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5"] Oct 30 20:00:24.805999 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:24.805263 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0bb3aef_a539_47c9_9aa5_b989236c6a7e.slice/crio-94d5acdf99117afcb719b84028537ce347b2eb96888d2cdbbc66b06fecb4b422 WatchSource:0}: Error finding container 94d5acdf99117afcb719b84028537ce347b2eb96888d2cdbbc66b06fecb4b422: Status 404 returned error can't find the container with id 94d5acdf99117afcb719b84028537ce347b2eb96888d2cdbbc66b06fecb4b422 Oct 30 20:00:24.809001 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.808939 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:24.809085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.809038 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:24.809085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.809073 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:24.850721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.850694 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:24.892101 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:24.892070 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:25.061538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.061501 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd"] Oct 30 20:00:25.064092 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:25.064056 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e7c420_1e78_42cf_abc6_25366f1e9387.slice/crio-0a5d7973129f939ac66db27f751abedd8fa30fd0315e54587c6fa3420451ef25 WatchSource:0}: Error finding container 0a5d7973129f939ac66db27f751abedd8fa30fd0315e54587c6fa3420451ef25: Status 404 returned error can't find the container with id 0a5d7973129f939ac66db27f751abedd8fa30fd0315e54587c6fa3420451ef25 Oct 30 20:00:25.067132 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.067093 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:25.067266 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.067178 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:25.067266 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.067221 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:25.151007 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.150950 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:25.222025 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.221992 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2"] Oct 30 20:00:25.226332 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:25.226285 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71651b4_40f9_4900_aa6d_84328edffa6d.slice/crio-ab90c0d42d3adb6beba7a7b4e90d55069eaf77d0af70ef3c5b9713d5d208af92 WatchSource:0}: Error finding container ab90c0d42d3adb6beba7a7b4e90d55069eaf77d0af70ef3c5b9713d5d208af92: Status 404 returned error can't find the container with id ab90c0d42d3adb6beba7a7b4e90d55069eaf77d0af70ef3c5b9713d5d208af92 Oct 30 20:00:25.228707 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.228672 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:25.228820 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.228751 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:25.228820 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.228794 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:25.644523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.644386 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" event={"ID":"a0bb3aef-a539-47c9-9aa5-b989236c6a7e","Type":"ContainerStarted","Data":"10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116"} Oct 30 20:00:25.645050 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.644559 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" event={"ID":"a0bb3aef-a539-47c9-9aa5-b989236c6a7e","Type":"ContainerStarted","Data":"94d5acdf99117afcb719b84028537ce347b2eb96888d2cdbbc66b06fecb4b422"} Oct 30 20:00:25.649319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.648277 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" event={"ID":"b71651b4-40f9-4900-aa6d-84328edffa6d","Type":"ContainerStarted","Data":"c2318339e9b73f9893c7f24eb4e1d5416cca1a3d3f3e93ee6cf4efa9b38a9598"} Oct 30 20:00:25.649319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.648333 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" event={"ID":"b71651b4-40f9-4900-aa6d-84328edffa6d","Type":"ContainerStarted","Data":"ab90c0d42d3adb6beba7a7b4e90d55069eaf77d0af70ef3c5b9713d5d208af92"} Oct 30 20:00:25.651568 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.651532 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" event={"ID":"85e7c420-1e78-42cf-abc6-25366f1e9387","Type":"ContainerStarted","Data":"868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01"} Oct 30 20:00:25.651686 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.651580 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" event={"ID":"85e7c420-1e78-42cf-abc6-25366f1e9387","Type":"ContainerStarted","Data":"0a5d7973129f939ac66db27f751abedd8fa30fd0315e54587c6fa3420451ef25"} Oct 30 20:00:25.699696 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.699633 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podStartSLOduration=2.699615268 podStartE2EDuration="2.699615268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:25.698649714 +0000 UTC m=+742.098821746" watchObservedRunningTime="2026-10-30 20:00:25.699615268 +0000 UTC m=+742.099787270" Oct 30 20:00:25.738062 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.738030 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:25.753126 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.751297 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" podStartSLOduration=1.751276416 podStartE2EDuration="1.751276416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:25.748326045 +0000 UTC m=+742.148498053" watchObservedRunningTime="2026-10-30 20:00:25.751276416 +0000 UTC m=+742.151448424" Oct 30 20:00:25.800829 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.800772 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podStartSLOduration=1.800751722 podStartE2EDuration="1.800751722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:25.799883364 +0000 UTC m=+742.200055371" watchObservedRunningTime="2026-10-30 20:00:25.800751722 +0000 UTC m=+742.200923793" Oct 30 20:00:25.892744 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:25.892705 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:26.484224 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:26.484182 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:26.491242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:26.491204 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:26.660724 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:26.660688 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:26.661368 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:26.661344 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:00:26.744689 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:26.744611 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:26.898386 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:26.898357 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:27.665423 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:27.665390 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:27.665869 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:27.665457 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:27.666146 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:27.666129 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:27.666227 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:27.666184 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:00:28.034619 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:28.034546 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:28.240244 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:28.240211 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:28.432269 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:28.432232 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:28.734461 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:28.734366 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:28.940970 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:28.940937 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:29.151937 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:29.151883 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:29.370028 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:29.369992 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:30.112095 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.112062 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km"] Oct 30 20:00:30.113108 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.113076 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:30.116147 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.116125 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.147510 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.147479 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-626870-serving-cert\"" Oct 30 20:00:30.149370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.149341 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km"] Oct 30 20:00:30.231910 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.231858 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda073ed-6c46-42ef-a592-c99c980f8885-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.232085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.232053 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-maas-api-tls\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.232127 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.232097 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tgtx\" (UniqueName: \"kubernetes.io/projected/fda073ed-6c46-42ef-a592-c99c980f8885-kube-api-access-8tgtx\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.232196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.232180 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-metrics-certs\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.310140 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.310106 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:30.333197 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.333155 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda073ed-6c46-42ef-a592-c99c980f8885-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.333372 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.333301 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-maas-api-tls\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.333372 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.333346 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tgtx\" (UniqueName: \"kubernetes.io/projected/fda073ed-6c46-42ef-a592-c99c980f8885-kube-api-access-8tgtx\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.333484 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.333394 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-metrics-certs\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.333947 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.333923 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda073ed-6c46-42ef-a592-c99c980f8885-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.336150 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.336126 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-maas-api-tls\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.336356 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.336337 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-metrics-certs\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.358852 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.358824 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tgtx\" (UniqueName: \"kubernetes.io/projected/fda073ed-6c46-42ef-a592-c99c980f8885-kube-api-access-8tgtx\") pod \"maas-api-e2e-worker-w5-626870-79795b9b56-hv9km\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.386106 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.382706 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c"] Oct 30 20:00:30.388449 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.388419 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:30.410282 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.410253 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-626870-dockercfg-cnc7f\"" Oct 30 20:00:30.410629 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.410590 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-626870\"" Oct 30 20:00:30.422984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.422952 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c"] Oct 30 20:00:30.433378 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.433343 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:30.434427 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.434397 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh5lw\" (UniqueName: \"kubernetes.io/projected/5c095837-e094-43c1-aa14-5715094b2bbb-kube-api-access-wh5lw\") pod \"payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c\" (UID: \"5c095837-e094-43c1-aa14-5715094b2bbb\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:30.434569 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.434440 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c095837-e094-43c1-aa14-5715094b2bbb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c\" (UID: \"5c095837-e094-43c1-aa14-5715094b2bbb\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:30.524777 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.524748 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:30.536199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.536164 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh5lw\" (UniqueName: \"kubernetes.io/projected/5c095837-e094-43c1-aa14-5715094b2bbb-kube-api-access-wh5lw\") pod \"payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c\" (UID: \"5c095837-e094-43c1-aa14-5715094b2bbb\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:30.536397 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.536222 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c095837-e094-43c1-aa14-5715094b2bbb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c\" (UID: \"5c095837-e094-43c1-aa14-5715094b2bbb\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:30.536942 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.536918 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c095837-e094-43c1-aa14-5715094b2bbb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c\" (UID: \"5c095837-e094-43c1-aa14-5715094b2bbb\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:30.561968 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.561943 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh5lw\" (UniqueName: \"kubernetes.io/projected/5c095837-e094-43c1-aa14-5715094b2bbb-kube-api-access-wh5lw\") pod \"payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c\" (UID: \"5c095837-e094-43c1-aa14-5715094b2bbb\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:30.709915 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.709817 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:30.739181 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:30.739151 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km"] Oct 30 20:00:30.741503 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:30.741469 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda073ed_6c46_42ef_a592_c99c980f8885.slice/crio-5bc226a5220dc5243c0da239f56868782f77f7d786a8c84fd8617cf4f58beab3 WatchSource:0}: Error finding container 5bc226a5220dc5243c0da239f56868782f77f7d786a8c84fd8617cf4f58beab3: Status 404 returned error can't find the container with id 5bc226a5220dc5243c0da239f56868782f77f7d786a8c84fd8617cf4f58beab3 Oct 30 20:00:31.012607 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.012571 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c"] Oct 30 20:00:31.014786 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:31.014747 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c095837_e094_43c1_aa14_5715094b2bbb.slice/crio-e2b5cd1d706f62551a4da8290b9eedbe663536daf9a9c4acf93abf5b50c96c2a WatchSource:0}: Error finding container e2b5cd1d706f62551a4da8290b9eedbe663536daf9a9c4acf93abf5b50c96c2a: Status 404 returned error can't find the container with id e2b5cd1d706f62551a4da8290b9eedbe663536daf9a9c4acf93abf5b50c96c2a Oct 30 20:00:31.686289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.686249 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" event={"ID":"fda073ed-6c46-42ef-a592-c99c980f8885","Type":"ContainerStarted","Data":"f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137"} Oct 30 20:00:31.686736 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.686383 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:31.686736 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.686429 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" event={"ID":"fda073ed-6c46-42ef-a592-c99c980f8885","Type":"ContainerStarted","Data":"5bc226a5220dc5243c0da239f56868782f77f7d786a8c84fd8617cf4f58beab3"} Oct 30 20:00:31.687746 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.687726 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" event={"ID":"5c095837-e094-43c1-aa14-5715094b2bbb","Type":"ContainerStarted","Data":"57a6fe4b1a817535b7bf93e49682a4253f5405cb800c8a97c37695c1df515211"} Oct 30 20:00:31.687863 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.687761 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" event={"ID":"5c095837-e094-43c1-aa14-5715094b2bbb","Type":"ContainerStarted","Data":"e2b5cd1d706f62551a4da8290b9eedbe663536daf9a9c4acf93abf5b50c96c2a"} Oct 30 20:00:31.687921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.687859 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:31.735546 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.735496 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" podStartSLOduration=1.37692935 podStartE2EDuration="1.735483051s" totalImagesPullingTime="358.553701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:31.733883806 +0000 UTC m=+748.134055816" watchObservedRunningTime="2026-10-30 20:00:31.735483051 +0000 UTC m=+748.135655058" Oct 30 20:00:31.778399 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.778345 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" podStartSLOduration=1.7783297070000001 podStartE2EDuration="1.778329707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:31.776995287 +0000 UTC m=+748.177167292" watchObservedRunningTime="2026-10-30 20:00:31.778329707 +0000 UTC m=+748.178501714" Oct 30 20:00:31.916962 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:31.916928 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:32.114818 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.114734 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:32.405940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.405885 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr"] Oct 30 20:00:32.409658 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.409634 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.429707 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.429682 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-efbac3-serving-cert\"" Oct 30 20:00:32.441808 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.441773 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr"] Oct 30 20:00:32.457112 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.457075 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-maas-api-tls\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.457283 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.457162 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slrfh\" (UniqueName: \"kubernetes.io/projected/b0c2935f-803d-4343-94a7-af159d12e098-kube-api-access-slrfh\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.457283 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.457218 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-metrics-certs\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.457418 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.457403 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c2935f-803d-4343-94a7-af159d12e098-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.485194 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.485165 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:32.558143 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.558109 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-maas-api-tls\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.558368 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.558180 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slrfh\" (UniqueName: \"kubernetes.io/projected/b0c2935f-803d-4343-94a7-af159d12e098-kube-api-access-slrfh\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.558368 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.558242 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-metrics-certs\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.558368 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.558352 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c2935f-803d-4343-94a7-af159d12e098-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.559146 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.559113 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c2935f-803d-4343-94a7-af159d12e098-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.560966 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.560940 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-maas-api-tls\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.561110 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.561091 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-metrics-certs\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.585027 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.584996 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slrfh\" (UniqueName: \"kubernetes.io/projected/b0c2935f-803d-4343-94a7-af159d12e098-kube-api-access-slrfh\") pod \"maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.656974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.656873 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:32.723252 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.723207 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:32.978921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:32.978811 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:33.022829 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:33.022795 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr"] Oct 30 20:00:33.031110 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:33.031080 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c2935f_803d_4343_94a7_af159d12e098.slice/crio-1f2583b6cb32736bc07d65d1910ede77b338556d9b292fc204409eead7bd7b54 WatchSource:0}: Error finding container 1f2583b6cb32736bc07d65d1910ede77b338556d9b292fc204409eead7bd7b54: Status 404 returned error can't find the container with id 1f2583b6cb32736bc07d65d1910ede77b338556d9b292fc204409eead7bd7b54 Oct 30 20:00:33.173815 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:33.173779 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:33.698857 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:33.698771 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" event={"ID":"b0c2935f-803d-4343-94a7-af159d12e098","Type":"ContainerStarted","Data":"4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359"} Oct 30 20:00:33.698857 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:33.698830 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" event={"ID":"b0c2935f-803d-4343-94a7-af159d12e098","Type":"ContainerStarted","Data":"1f2583b6cb32736bc07d65d1910ede77b338556d9b292fc204409eead7bd7b54"} Oct 30 20:00:33.699141 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:33.698960 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:33.745412 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:33.745360 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" podStartSLOduration=1.363433256 podStartE2EDuration="1.745344584s" totalImagesPullingTime="381.911328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:33.743704996 +0000 UTC m=+750.143877001" watchObservedRunningTime="2026-10-30 20:00:33.745344584 +0000 UTC m=+750.145516590" Oct 30 20:00:34.462015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:34.461986 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:35.092271 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:00:35.092235 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:00:35.528559 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.528523 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:35.708948 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.708886 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:35.709127 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.709110 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk"] Oct 30 20:00:35.712752 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.712730 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:35.730247 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.730229 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-62f4a1\"" Oct 30 20:00:35.741226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.741201 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-62f4a1-dockercfg-pfxkm\"" Oct 30 20:00:35.743930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.743890 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk"] Oct 30 20:00:35.792890 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.792807 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aeddd424-8ad6-40dd-aaec-68195c43415e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk\" (UID: \"aeddd424-8ad6-40dd-aaec-68195c43415e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:35.793060 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.793035 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhp94\" (UniqueName: \"kubernetes.io/projected/aeddd424-8ad6-40dd-aaec-68195c43415e-kube-api-access-zhp94\") pod \"payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk\" (UID: \"aeddd424-8ad6-40dd-aaec-68195c43415e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:35.894224 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.894186 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aeddd424-8ad6-40dd-aaec-68195c43415e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk\" (UID: \"aeddd424-8ad6-40dd-aaec-68195c43415e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:35.894399 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.894289 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhp94\" (UniqueName: \"kubernetes.io/projected/aeddd424-8ad6-40dd-aaec-68195c43415e-kube-api-access-zhp94\") pod \"payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk\" (UID: \"aeddd424-8ad6-40dd-aaec-68195c43415e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:35.894877 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.894859 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aeddd424-8ad6-40dd-aaec-68195c43415e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk\" (UID: \"aeddd424-8ad6-40dd-aaec-68195c43415e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:35.920086 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:35.920053 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhp94\" (UniqueName: \"kubernetes.io/projected/aeddd424-8ad6-40dd-aaec-68195c43415e-kube-api-access-zhp94\") pod \"payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk\" (UID: \"aeddd424-8ad6-40dd-aaec-68195c43415e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:36.026303 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.026261 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:36.210299 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.210264 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2"] Oct 30 20:00:36.217106 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.216281 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.236217 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.236189 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2"] Oct 30 20:00:36.238937 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.238913 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-912f24-serving-cert\"" Oct 30 20:00:36.288792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.288756 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:36.304003 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.303974 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92199c-6d6c-4020-b877-8e585a87cd28-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.304146 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.304016 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.304218 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.304184 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-metrics-certs\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.304218 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.304210 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w4wf\" (UniqueName: \"kubernetes.io/projected/4a92199c-6d6c-4020-b877-8e585a87cd28-kube-api-access-5w4wf\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.340882 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.340860 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk"] Oct 30 20:00:36.343064 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:36.343026 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeddd424_8ad6_40dd_aaec_68195c43415e.slice/crio-0b8ab6561806c4e9e5ad9573bd53f2fa41deb5a7d060f770e8210a4715583719 WatchSource:0}: Error finding container 0b8ab6561806c4e9e5ad9573bd53f2fa41deb5a7d060f770e8210a4715583719: Status 404 returned error can't find the container with id 0b8ab6561806c4e9e5ad9573bd53f2fa41deb5a7d060f770e8210a4715583719 Oct 30 20:00:36.405589 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.405559 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92199c-6d6c-4020-b877-8e585a87cd28-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.405712 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.405624 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.405770 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.405746 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-metrics-certs\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.405830 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.405776 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w4wf\" (UniqueName: \"kubernetes.io/projected/4a92199c-6d6c-4020-b877-8e585a87cd28-kube-api-access-5w4wf\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.405889 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:00:36.405837 6411 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-serving-cert: secret "maas-api-e2e-worker-w3-912f24-serving-cert" not found Oct 30 20:00:36.405976 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:00:36.405956 6411 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls podName:4a92199c-6d6c-4020-b877-8e585a87cd28 nodeName:}" failed. No retries permitted until 2026-10-30 20:00:36.905932763 +0000 UTC m=+753.306104750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls") pod "maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" (UID: "4a92199c-6d6c-4020-b877-8e585a87cd28") : secret "maas-api-e2e-worker-w3-912f24-serving-cert" not found Oct 30 20:00:36.406269 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.406251 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92199c-6d6c-4020-b877-8e585a87cd28-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.408704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.408680 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-metrics-certs\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.432811 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.432776 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w4wf\" (UniqueName: \"kubernetes.io/projected/4a92199c-6d6c-4020-b877-8e585a87cd28-kube-api-access-5w4wf\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.474626 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.474590 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:36.661719 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.661682 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:36.716578 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.716534 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" event={"ID":"aeddd424-8ad6-40dd-aaec-68195c43415e","Type":"ContainerStarted","Data":"0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569"} Oct 30 20:00:36.716763 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.716607 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" event={"ID":"aeddd424-8ad6-40dd-aaec-68195c43415e","Type":"ContainerStarted","Data":"0b8ab6561806c4e9e5ad9573bd53f2fa41deb5a7d060f770e8210a4715583719"} Oct 30 20:00:36.716763 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.716640 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:36.768862 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.768816 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" podStartSLOduration=1.76880105 podStartE2EDuration="1.76880105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:36.767428514 +0000 UTC m=+753.167600522" watchObservedRunningTime="2026-10-30 20:00:36.76880105 +0000 UTC m=+753.168973064" Oct 30 20:00:36.911486 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.911450 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:36.914254 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:36.914198 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls\") pod \"maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:37.132020 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:37.131977 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:37.432724 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:37.432699 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2"] Oct 30 20:00:37.435845 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:37.435781 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a92199c_6d6c_4020_b877_8e585a87cd28.slice/crio-e0d12e401e1411ea39ae4bccc5350c154ebe34da13e7f1652a5b87b68ae49f30 WatchSource:0}: Error finding container e0d12e401e1411ea39ae4bccc5350c154ebe34da13e7f1652a5b87b68ae49f30: Status 404 returned error can't find the container with id e0d12e401e1411ea39ae4bccc5350c154ebe34da13e7f1652a5b87b68ae49f30 Oct 30 20:00:37.614308 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:37.614276 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:37.698209 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:37.698139 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:00:37.722778 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:37.722744 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" event={"ID":"4a92199c-6d6c-4020-b877-8e585a87cd28","Type":"ContainerStarted","Data":"e0d12e401e1411ea39ae4bccc5350c154ebe34da13e7f1652a5b87b68ae49f30"} Oct 30 20:00:37.794874 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:37.794838 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:38.204408 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.204364 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5"] Oct 30 20:00:38.209008 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.208928 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.234154 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.231252 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5"] Oct 30 20:00:38.234154 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.231511 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-62f4a1-serving-cert\"" Oct 30 20:00:38.282843 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.282803 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:38.326689 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.326658 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-maas-api-tls\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.326689 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.326694 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4dca37-ec59-4707-bd27-0eb7d6b556af-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.326974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.326781 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-metrics-certs\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.326974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.326957 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vnd6\" (UniqueName: \"kubernetes.io/projected/fc4dca37-ec59-4707-bd27-0eb7d6b556af-kube-api-access-4vnd6\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.428529 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.428496 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-maas-api-tls\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.428529 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.428532 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4dca37-ec59-4707-bd27-0eb7d6b556af-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.428783 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.428585 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-metrics-certs\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.428783 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.428655 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vnd6\" (UniqueName: \"kubernetes.io/projected/fc4dca37-ec59-4707-bd27-0eb7d6b556af-kube-api-access-4vnd6\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.429358 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.429321 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4dca37-ec59-4707-bd27-0eb7d6b556af-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.431061 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.431040 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-maas-api-tls\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.431244 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.431222 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-metrics-certs\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.464154 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.464096 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vnd6\" (UniqueName: \"kubernetes.io/projected/fc4dca37-ec59-4707-bd27-0eb7d6b556af-kube-api-access-4vnd6\") pod \"maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.500912 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.500854 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:38.527099 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.527070 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:38.730041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.729944 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" event={"ID":"4a92199c-6d6c-4020-b877-8e585a87cd28","Type":"ContainerStarted","Data":"2d02b1184980bfd8e5441d9424849aaf0a39412c9a68b39d054ee2ed3219d2c6"} Oct 30 20:00:38.730041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.730024 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:38.779607 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.779549 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" podStartSLOduration=2.348668793 podStartE2EDuration="2.779533939s" totalImagesPullingTime="430.865146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:38.777148872 +0000 UTC m=+755.177320879" watchObservedRunningTime="2026-10-30 20:00:38.779533939 +0000 UTC m=+755.179705946" Oct 30 20:00:38.865382 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:38.865343 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5"] Oct 30 20:00:38.868283 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:38.868244 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4dca37_ec59_4707_bd27_0eb7d6b556af.slice/crio-98380ed1d254e0326b3242695e144b268c4777568461b0d30d243cc803cd2168 WatchSource:0}: Error finding container 98380ed1d254e0326b3242695e144b268c4777568461b0d30d243cc803cd2168: Status 404 returned error can't find the container with id 98380ed1d254e0326b3242695e144b268c4777568461b0d30d243cc803cd2168 Oct 30 20:00:39.541221 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:39.541117 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:39.711173 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:39.711139 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:00:39.743559 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:39.742533 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:39.749683 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:39.749650 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" event={"ID":"fc4dca37-ec59-4707-bd27-0eb7d6b556af","Type":"ContainerStarted","Data":"36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288"} Oct 30 20:00:39.749833 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:39.749716 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" event={"ID":"fc4dca37-ec59-4707-bd27-0eb7d6b556af","Type":"ContainerStarted","Data":"98380ed1d254e0326b3242695e144b268c4777568461b0d30d243cc803cd2168"} Oct 30 20:00:39.831983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:39.830764 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" podStartSLOduration=1.428269655 podStartE2EDuration="1.830742873s" totalImagesPullingTime="402.473218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:39.828646082 +0000 UTC m=+756.228818091" watchObservedRunningTime="2026-10-30 20:00:39.830742873 +0000 UTC m=+756.230914882" Oct 30 20:00:40.552096 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:40.552062 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:40.747204 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:40.747166 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:40.754615 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:40.754589 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:42.227496 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.227462 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r"] Oct 30 20:00:42.232441 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.232415 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.253209 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.253158 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r"] Oct 30 20:00:42.267270 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.265179 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-f1f016-serving-cert\"" Oct 30 20:00:42.290985 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.290943 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.291375 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.291337 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-metrics-certs\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.291599 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.291579 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctqvk\" (UniqueName: \"kubernetes.io/projected/b39478e6-123e-48ad-b7f4-d48da825d8ee-kube-api-access-ctqvk\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.304968 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.291952 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39478e6-123e-48ad-b7f4-d48da825d8ee-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.325418 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.324956 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:42.413951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.412774 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39478e6-123e-48ad-b7f4-d48da825d8ee-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.414295 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.414219 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.414463 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.414429 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-metrics-certs\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.414546 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.414502 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctqvk\" (UniqueName: \"kubernetes.io/projected/b39478e6-123e-48ad-b7f4-d48da825d8ee-kube-api-access-ctqvk\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.414682 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.414063 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39478e6-123e-48ad-b7f4-d48da825d8ee-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.415143 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:00:42.414337 6411 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-serving-cert: secret "maas-api-e2e-worker-w4-f1f016-serving-cert" not found Oct 30 20:00:42.415247 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:00:42.415219 6411 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls podName:b39478e6-123e-48ad-b7f4-d48da825d8ee nodeName:}" failed. No retries permitted until 2026-10-30 20:00:42.915196625 +0000 UTC m=+759.315368614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls") pod "maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" (UID: "b39478e6-123e-48ad-b7f4-d48da825d8ee") : secret "maas-api-e2e-worker-w4-f1f016-serving-cert" not found Oct 30 20:00:42.418144 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.418115 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-metrics-certs\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.442423 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.441485 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctqvk\" (UniqueName: \"kubernetes.io/projected/b39478e6-123e-48ad-b7f4-d48da825d8ee-kube-api-access-ctqvk\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.501736 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.501636 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:42.694873 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.694840 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:42.710411 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.710383 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:00:42.922617 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.922571 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:42.925355 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:42.925326 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls\") pod \"maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:43.153691 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:43.153657 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:43.506370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:43.506239 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r"] Oct 30 20:00:43.508823 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:43.508769 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39478e6_123e_48ad_b7f4_d48da825d8ee.slice/crio-3429d30b6124dec044317fcfcd01e4d4c880c3b37e49a0f3573c75af998452d8 WatchSource:0}: Error finding container 3429d30b6124dec044317fcfcd01e4d4c880c3b37e49a0f3573c75af998452d8: Status 404 returned error can't find the container with id 3429d30b6124dec044317fcfcd01e4d4c880c3b37e49a0f3573c75af998452d8 Oct 30 20:00:43.805160 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:43.805051 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" event={"ID":"b39478e6-123e-48ad-b7f4-d48da825d8ee","Type":"ContainerStarted","Data":"3429d30b6124dec044317fcfcd01e4d4c880c3b37e49a0f3573c75af998452d8"} Oct 30 20:00:44.210950 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:44.210882 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:44.755558 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:44.755531 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:00:44.810978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:44.810931 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" event={"ID":"b39478e6-123e-48ad-b7f4-d48da825d8ee","Type":"ContainerStarted","Data":"4b224ced077af21dae5dcd1bf9412a544ba0b824a6c0414cc844bf9c02e5d960"} Oct 30 20:00:44.811193 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:44.811047 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:44.859095 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:44.859038 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" podStartSLOduration=2.225417624 podStartE2EDuration="2.859016702s" totalImagesPullingTime="633.599078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:44.856838404 +0000 UTC m=+761.257010413" watchObservedRunningTime="2026-10-30 20:00:44.859016702 +0000 UTC m=+761.259188710" Oct 30 20:00:46.029014 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:46.028978 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:46.809923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:46.809344 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:00:47.748056 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:47.746768 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:00:48.170343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:48.170310 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:48.354524 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:48.354440 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:48.635011 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:48.609750 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:48.896251 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:48.896166 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:49.218853 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:49.218721 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:50.233557 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:50.233517 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:50.848934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:50.848497 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:00:52.167763 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:52.167686 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:52.348135 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:52.348082 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:53.362835 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:53.362802 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:53.989784 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:53.989755 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:54.047917 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.047038 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x"] Oct 30 20:00:54.055785 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.055757 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.095498 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.094161 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x"] Oct 30 20:00:54.175316 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175272 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.175523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175369 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.175523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175394 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28159db6-1d64-4199-8833-06c5eba271e1-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.175523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175427 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.175523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175456 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28159db6-1d64-4199-8833-06c5eba271e1-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.175523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175485 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.175523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175522 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.175855 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175603 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.175855 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.175677 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdj4w\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-kube-api-access-bdj4w\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.198836 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.198726 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:54.276543 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276433 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.276543 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276498 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.276543 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276522 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28159db6-1d64-4199-8833-06c5eba271e1-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.276828 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276552 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.276828 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276578 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28159db6-1d64-4199-8833-06c5eba271e1-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.276828 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276605 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.276828 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276658 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.276828 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276742 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.276828 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.276825 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdj4w\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-kube-api-access-bdj4w\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.277151 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.277027 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.277817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.277420 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.277817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.277513 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.277817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.277632 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28159db6-1d64-4199-8833-06c5eba271e1-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.278097 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.277847 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.279727 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.279704 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28159db6-1d64-4199-8833-06c5eba271e1-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.282165 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.282140 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.306435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.305227 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.306435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.305442 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdj4w\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-kube-api-access-bdj4w\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.396164 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.396125 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:54.748125 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.748031 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x"] Oct 30 20:00:54.758713 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:00:54.752355 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28159db6_1d64_4199_8833_06c5eba271e1.slice/crio-e8f5c8088dfb12b5da5c4525e7f00f8743ee536023ba199659c9a0c9bf3f7784 WatchSource:0}: Error finding container e8f5c8088dfb12b5da5c4525e7f00f8743ee536023ba199659c9a0c9bf3f7784: Status 404 returned error can't find the container with id e8f5c8088dfb12b5da5c4525e7f00f8743ee536023ba199659c9a0c9bf3f7784 Oct 30 20:00:54.758713 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.756925 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:54.758713 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.757039 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:54.758713 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.757122 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:54.980006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.979961 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" event={"ID":"28159db6-1d64-4199-8833-06c5eba271e1","Type":"ContainerStarted","Data":"65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350"} Oct 30 20:00:54.980178 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.980040 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" event={"ID":"28159db6-1d64-4199-8833-06c5eba271e1","Type":"ContainerStarted","Data":"e8f5c8088dfb12b5da5c4525e7f00f8743ee536023ba199659c9a0c9bf3f7784"} Oct 30 20:00:54.982660 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.982625 6411 generic.go:356] "Generic (PLEG): container finished" podID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerID="c2318339e9b73f9893c7f24eb4e1d5416cca1a3d3f3e93ee6cf4efa9b38a9598" exitCode=0 Oct 30 20:00:54.982798 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.982666 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" event={"ID":"b71651b4-40f9-4900-aa6d-84328edffa6d","Type":"ContainerDied","Data":"c2318339e9b73f9893c7f24eb4e1d5416cca1a3d3f3e93ee6cf4efa9b38a9598"} Oct 30 20:00:54.983024 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.982995 6411 scope.go:122] "RemoveContainer" containerID="c2318339e9b73f9893c7f24eb4e1d5416cca1a3d3f3e93ee6cf4efa9b38a9598" Oct 30 20:00:54.984494 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.984459 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:54.984598 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.984531 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:54.984598 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:54.984577 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:55.105946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:55.098351 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podStartSLOduration=1.098330595 podStartE2EDuration="1.098330595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:55.042240769 +0000 UTC m=+771.442412815" watchObservedRunningTime="2026-10-30 20:00:55.098330595 +0000 UTC m=+771.498502603" Oct 30 20:00:55.402924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:55.401788 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:00:55.402924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:55.402048 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Oct 30 20:00:55.402924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:55.402128 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Oct 30 20:00:56.030922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:56.018004 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" event={"ID":"b71651b4-40f9-4900-aa6d-84328edffa6d","Type":"ContainerStarted","Data":"10c655f4708918b7579b9eaf436248b2c9cbbe159d86220ca8845a4520dba3e3"} Oct 30 20:00:56.358331 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:56.358219 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:56.400503 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:56.400468 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Oct 30 20:00:56.400840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:56.400787 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Oct 30 20:00:56.897924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:56.895222 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:56.897924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:56.895354 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Oct 30 20:00:56.897924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:56.895398 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Oct 30 20:00:57.398605 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:57.398564 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Oct 30 20:00:57.398950 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:57.398857 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Oct 30 20:00:57.665340 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:57.665258 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:00:57.729942 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:57.704967 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:00:57.729942 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:57.705064 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:00:57.897924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:57.893058 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Oct 30 20:00:57.897924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:57.893120 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Oct 30 20:00:58.411041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:58.401332 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Oct 30 20:00:58.411041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:58.401401 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Oct 30 20:00:58.458927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:58.449720 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:58.644955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:58.644690 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": read tcp 10.133.0.2:51490->10.133.0.44:15021: read: connection reset by peer" start-of-body= Oct 30 20:00:58.644955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:58.644754 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": read tcp 10.133.0.2:51490->10.133.0.44:15021: read: connection reset by peer" Oct 30 20:00:58.650170 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:58.650129 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:58.895767 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:58.895325 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Oct 30 20:00:58.895767 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:58.895382 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Oct 30 20:00:59.033666 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.033483 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:59.044614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.044588 6411 generic.go:356] "Generic (PLEG): container finished" podID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerID="10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116" exitCode=0 Oct 30 20:00:59.044749 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.044721 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" event={"ID":"a0bb3aef-a539-47c9-9aa5-b989236c6a7e","Type":"ContainerDied","Data":"10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116"} Oct 30 20:00:59.045458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.045155 6411 scope.go:122] "RemoveContainer" containerID="10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116" Oct 30 20:00:59.046671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.046641 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:59.046729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.046711 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:59.046772 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.046744 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:59.047460 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.047441 6411 generic.go:356] "Generic (PLEG): container finished" podID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerID="868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01" exitCode=0 Oct 30 20:00:59.047542 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.047501 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" event={"ID":"85e7c420-1e78-42cf-abc6-25366f1e9387","Type":"ContainerDied","Data":"868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01"} Oct 30 20:00:59.047819 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.047803 6411 scope.go:122] "RemoveContainer" containerID="868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01" Oct 30 20:00:59.049446 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.049401 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:59.049680 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.049657 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:59.049758 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.049717 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:00:59.255201 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.255171 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:59.406917 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.404562 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Oct 30 20:00:59.406917 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.404621 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Oct 30 20:00:59.496960 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.478347 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:00:59.896376 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.896333 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Oct 30 20:00:59.896652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:00:59.896622 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Oct 30 20:01:00.067813 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.067705 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" event={"ID":"85e7c420-1e78-42cf-abc6-25366f1e9387","Type":"ContainerStarted","Data":"b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf"} Oct 30 20:01:00.070361 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.070335 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" event={"ID":"a0bb3aef-a539-47c9-9aa5-b989236c6a7e","Type":"ContainerStarted","Data":"e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f"} Oct 30 20:01:00.400466 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.400430 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Oct 30 20:01:00.400773 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.400727 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Oct 30 20:01:00.486450 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.486409 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:01:00.486837 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.486800 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Oct 30 20:01:00.487068 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.487034 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Oct 30 20:01:00.740459 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.740369 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:01:00.741532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.741424 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 20:01:00.741730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.741707 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 20:01:00.905573 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.905529 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Oct 30 20:01:00.930978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:00.905828 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Oct 30 20:01:01.124475 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:01.124342 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:01.407923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:01.400460 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Oct 30 20:01:01.407923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:01.400539 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Oct 30 20:01:01.484507 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:01.484470 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Oct 30 20:01:01.484812 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:01.484765 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Oct 30 20:01:01.739703 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:01.739616 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 20:01:01.741438 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:01.741405 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 20:01:02.408313 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.408265 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:01:02.410507 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.410480 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:01:02.417669 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.417645 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:01:02.423114 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.423094 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:01:02.423522 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.423506 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:01:02.489390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.489359 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Oct 30 20:01:02.489680 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.489643 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Oct 30 20:01:02.759226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.740297 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 20:01:02.759226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:02.740398 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 20:01:03.268946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.268909 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:03.485369 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.485329 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Oct 30 20:01:03.485983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.485950 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Oct 30 20:01:03.737943 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.737874 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 20:01:03.738139 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.737991 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 20:01:03.828927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.804452 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs"] Oct 30 20:01:03.846541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.846495 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:03.856555 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.856534 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs"] Oct 30 20:01:03.856807 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.856766 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:03.873638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.873611 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-efbac3\"/\"kube-root-ca.crt\"" Oct 30 20:01:03.873813 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.873790 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-efbac3\"/\"simulator-w2-efbac3-kserve-self-signed-certs\"" Oct 30 20:01:03.873934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.873838 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-efbac3\"/\"default-dockercfg-j897c\"" Oct 30 20:01:03.873934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.873853 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-efbac3\"/\"openshift-service-ca.crt\"" Oct 30 20:01:03.954825 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.954801 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-home\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:03.955035 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.955019 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-dshm\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:03.955147 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.955135 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tls-certs\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:03.955283 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.955265 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tmp-dir\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:03.955345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.955325 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjdrv\" (UniqueName: \"kubernetes.io/projected/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-kube-api-access-hjdrv\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:03.955394 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:03.955364 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-model-cache\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.034208 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.034119 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:04.056004 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.055972 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-home\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.056197 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.056025 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-dshm\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.056197 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.056075 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tls-certs\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.056197 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.056133 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tmp-dir\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.056197 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.056158 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjdrv\" (UniqueName: \"kubernetes.io/projected/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-kube-api-access-hjdrv\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.056197 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.056195 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-model-cache\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.056878 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.056852 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-model-cache\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.057164 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.057145 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tmp-dir\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.057402 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.057384 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-home\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.059638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.059612 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tls-certs\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.060377 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.060353 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-dshm\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.095543 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.095473 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjdrv\" (UniqueName: \"kubernetes.io/projected/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-kube-api-access-hjdrv\") pod \"simulator-w2-efbac3-kserve-855cc55c8f-st9cs\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.175479 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.175439 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:04.523934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.515037 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Oct 30 20:01:04.523934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.515103 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Oct 30 20:01:04.523934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.515981 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:04.558701 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.558673 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs"] Oct 30 20:01:04.561066 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:01:04.561039 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6a26c55_a188_4bd8_ba15_60bb3a5deb2c.slice/crio-ae72b4ee386bdd442202d32bc027f023e89d682cc16955ff4a2a39a7bfaf23ca WatchSource:0}: Error finding container ae72b4ee386bdd442202d32bc027f023e89d682cc16955ff4a2a39a7bfaf23ca: Status 404 returned error can't find the container with id ae72b4ee386bdd442202d32bc027f023e89d682cc16955ff4a2a39a7bfaf23ca Oct 30 20:01:04.739923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.739355 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 20:01:04.739923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.739428 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 20:01:04.762921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:04.762364 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:05.152730 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:01:05.152687 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:01:05.174277 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:05.174191 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" event={"ID":"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c","Type":"ContainerStarted","Data":"551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844"} Oct 30 20:01:05.174277 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:05.174244 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" event={"ID":"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c","Type":"ContainerStarted","Data":"ae72b4ee386bdd442202d32bc027f023e89d682cc16955ff4a2a39a7bfaf23ca"} Oct 30 20:01:05.176450 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:05.176411 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:05.206092 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:05.206025 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:05.486423 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:05.486330 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Oct 30 20:01:05.486713 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:05.486666 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Oct 30 20:01:05.739440 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:05.739336 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 20:01:05.739440 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:05.739403 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 20:01:06.706731 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:06.706399 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:01:06.706969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:06.706953 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:01:06.708343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:06.708318 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:01:06.742823 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:06.742797 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:01:06.743215 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:06.743081 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:01:06.743766 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:06.743749 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:01:06.770728 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:06.770681 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podStartSLOduration=3.7706679039999997 podStartE2EDuration="3.770667904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:01:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:01:05.242556012 +0000 UTC m=+781.642728020" watchObservedRunningTime="2026-10-30 20:01:06.770667904 +0000 UTC m=+783.170839910" Oct 30 20:01:07.143872 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:07.143842 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:07.889297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:07.889258 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:09.248142 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:09.248106 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:09.658344 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:09.658316 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:09.938880 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:09.938805 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:10.265193 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:10.265102 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:10.468707 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:10.468674 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:11.154183 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:11.154151 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:12.664874 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:12.664833 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:13.152945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:13.152882 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:13.395431 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:13.395401 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:14.183556 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:14.182325 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:14.217273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:14.217244 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:14.250828 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:14.250772 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:01:15.094247 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:15.094211 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:16.894350 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:16.894313 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:18.407100 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:18.407061 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:19.029458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.029415 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:19.285751 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.285642 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:19.462312 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.462265 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:19.911864 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.911835 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c"] Oct 30 20:01:19.915753 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.915728 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:19.944540 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.944514 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5f10d2\"/\"simulator-w1-5f10d2-kserve-self-signed-certs\"" Oct 30 20:01:19.944641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.944564 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5f10d2\"/\"default-dockercfg-jk2qk\"" Oct 30 20:01:19.944641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.944591 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-5f10d2\"/\"openshift-service-ca.crt\"" Oct 30 20:01:19.945018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.944993 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-5f10d2\"/\"kube-root-ca.crt\"" Oct 30 20:01:19.947452 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.947431 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c"] Oct 30 20:01:19.958956 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:19.958918 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:20.077668 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.077637 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-home\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.077833 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.077738 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt8j5\" (UniqueName: \"kubernetes.io/projected/97a27bca-4f25-4da5-ad54-e4dab81fe99b-kube-api-access-jt8j5\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.077833 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.077772 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tls-certs\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.077976 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.077852 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-dshm\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.077976 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.077944 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tmp-dir\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.078078 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.078053 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-model-cache\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.179349 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt8j5\" (UniqueName: \"kubernetes.io/projected/97a27bca-4f25-4da5-ad54-e4dab81fe99b-kube-api-access-jt8j5\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.179400 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tls-certs\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.179438 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-dshm\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.179519 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tmp-dir\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.179579 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-model-cache\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.179693 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-home\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180449 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.180257 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tmp-dir\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180449 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.180335 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-home\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.180597 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.180574 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-model-cache\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.184547 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.184497 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-dshm\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.192494 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.192461 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tls-certs\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.207258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.207226 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt8j5\" (UniqueName: \"kubernetes.io/projected/97a27bca-4f25-4da5-ad54-e4dab81fe99b-kube-api-access-jt8j5\") pod \"simulator-w1-5f10d2-kserve-fd4877896-d765c\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.228538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.228515 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:20.561767 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.561742 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c"] Oct 30 20:01:20.563727 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:01:20.563699 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97a27bca_4f25_4da5_ad54_e4dab81fe99b.slice/crio-58d6f0eaf68055429bc34958eaa9b7677d97e0b60b80e6e4262c79edadc92321 WatchSource:0}: Error finding container 58d6f0eaf68055429bc34958eaa9b7677d97e0b60b80e6e4262c79edadc92321: Status 404 returned error can't find the container with id 58d6f0eaf68055429bc34958eaa9b7677d97e0b60b80e6e4262c79edadc92321 Oct 30 20:01:20.871600 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:20.871512 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:21.285050 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:21.285010 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" event={"ID":"97a27bca-4f25-4da5-ad54-e4dab81fe99b","Type":"ContainerStarted","Data":"ae629cf7663866dc4d7c0e560ec4b933ca15ebcedb145d6856f4dd5317962f30"} Oct 30 20:01:21.285220 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:21.285061 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" event={"ID":"97a27bca-4f25-4da5-ad54-e4dab81fe99b","Type":"ContainerStarted","Data":"58d6f0eaf68055429bc34958eaa9b7677d97e0b60b80e6e4262c79edadc92321"} Oct 30 20:01:21.331618 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:21.331562 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podStartSLOduration=2.331550288 podStartE2EDuration="2.331550288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:01:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:01:21.330945131 +0000 UTC m=+797.731117135" watchObservedRunningTime="2026-10-30 20:01:21.331550288 +0000 UTC m=+797.731722297" Oct 30 20:01:22.229351 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:22.229324 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:22.721155 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:22.721111 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:24.772184 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:24.772146 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:26.641166 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:26.641135 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:28.467330 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:28.467284 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:28.740490 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:28.740386 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:30.229505 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:30.229475 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:30.244711 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:30.244685 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:30.291460 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:30.291426 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:30.358797 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:30.358767 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:01:32.050677 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:32.050646 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:34.022936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:34.022878 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:34.451662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:34.451628 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:35.090927 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:01:35.090876 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:01:35.903402 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:35.903372 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:37.687220 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:37.687187 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:39.832861 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:39.832827 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:41.699870 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:41.699732 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:43.635728 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:43.635690 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:43.907004 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:43.906908 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:44.824870 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:44.824831 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:45.805810 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:45.805775 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:47.615323 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:47.615288 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:49.495549 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:49.495516 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:50.345313 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:50.345276 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:51.307809 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:51.307742 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:53.157872 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:53.157833 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:55.180795 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:55.180759 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:56.960756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:56.960717 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:58.158078 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.158050 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:58.585219 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.585125 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx"] Oct 30 20:01:58.589446 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.589420 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.607390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.607367 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-62f4a1\"/\"openshift-service-ca.crt\"" Oct 30 20:01:58.619513 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.619487 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-62f4a1\"/\"kube-root-ca.crt\"" Oct 30 20:01:58.619631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.619534 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-62f4a1\"/\"premium-w0-62f4a1-kserve-self-signed-certs\"" Oct 30 20:01:58.619939 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.619920 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-62f4a1\"/\"default-dockercfg-pv79q\"" Oct 30 20:01:58.623685 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.623660 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx"] Oct 30 20:01:58.642066 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.642042 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:58.742958 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.742933 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-dshm\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.743119 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.743065 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-home\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.743119 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.743113 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvnv4\" (UniqueName: \"kubernetes.io/projected/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-kube-api-access-cvnv4\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.743218 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.743206 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tmp-dir\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.743296 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.743282 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-model-cache\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.743341 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.743315 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tls-certs\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.844923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.844820 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-dshm\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.845103 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.844959 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-home\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.845103 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.845010 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvnv4\" (UniqueName: \"kubernetes.io/projected/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-kube-api-access-cvnv4\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.845103 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.845042 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tmp-dir\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.845285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.845141 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-model-cache\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.845285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.845205 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tls-certs\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.845577 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.845548 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tmp-dir\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.845577 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.845565 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-model-cache\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.845756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.845591 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-home\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.847444 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.847427 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-dshm\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.847958 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.847937 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tls-certs\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.871735 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.871715 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvnv4\" (UniqueName: \"kubernetes.io/projected/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-kube-api-access-cvnv4\") pod \"premium-w0-62f4a1-kserve-757d46bc8-fwgmx\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.903659 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.903635 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:01:58.964606 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:58.964571 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:01:59.209210 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:59.209183 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx"] Oct 30 20:01:59.211405 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:01:59.211382 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f3e0a8f_33c8_4a22_a9f8_d1cf75e60fca.slice/crio-dd1acac5a1cf2d3fe0d54b54f5cf2a919a48504016dd8bc58eb4b49b1e871458 WatchSource:0}: Error finding container dd1acac5a1cf2d3fe0d54b54f5cf2a919a48504016dd8bc58eb4b49b1e871458: Status 404 returned error can't find the container with id dd1acac5a1cf2d3fe0d54b54f5cf2a919a48504016dd8bc58eb4b49b1e871458 Oct 30 20:01:59.498041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:59.497945 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" event={"ID":"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca","Type":"ContainerStarted","Data":"52773eb19098882b93d93810587ada6953ff99a9ad679355e4634305eabafe74"} Oct 30 20:01:59.498041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:59.498014 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" event={"ID":"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca","Type":"ContainerStarted","Data":"dd1acac5a1cf2d3fe0d54b54f5cf2a919a48504016dd8bc58eb4b49b1e871458"} Oct 30 20:01:59.544843 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:59.544762 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podStartSLOduration=1.5447482529999998 podStartE2EDuration="1.544748253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:01:59.543126412 +0000 UTC m=+835.943298418" watchObservedRunningTime="2026-10-30 20:01:59.544748253 +0000 UTC m=+835.944920296" Oct 30 20:01:59.904005 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:01:59.903975 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:02:00.814185 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:00.814155 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:01.779558 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:01.779528 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:02.815618 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:02.815585 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:04.695264 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:04.695223 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:04.936126 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:04.936090 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:05.119935 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:02:05.119796 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:02:06.435261 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:06.435219 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:08.434575 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:08.434535 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:08.904461 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:08.904424 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:02:08.917783 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:08.917753 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:02:09.507094 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.507048 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h"] Oct 30 20:02:09.511325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.511305 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.529890 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.529858 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-efbac3-openshift-default-dockercfg-wbss2\"" Oct 30 20:02:09.555342 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.555306 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h"] Oct 30 20:02:09.572752 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.572726 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:02:09.587704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.587663 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738fae3-4f39-49cd-8368-02e1ea0134e9-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.587885 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.587802 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.587885 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.587872 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.588072 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.588014 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.588072 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.588057 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.588182 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.588088 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.588182 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.588115 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmbsp\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-kube-api-access-fmbsp\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.588393 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.588352 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.588530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.588459 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690177 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690136 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690373 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690188 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738fae3-4f39-49cd-8368-02e1ea0134e9-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690373 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690264 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690552 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690525 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690651 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690632 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690728 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690709 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690789 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690773 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690843 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690806 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690843 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690833 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.690980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690856 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbsp\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-kube-api-access-fmbsp\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.691074 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.690986 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.691074 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.691005 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.691074 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.691010 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738fae3-4f39-49cd-8368-02e1ea0134e9-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.691298 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.691276 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.693462 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.693434 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.694094 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.694063 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.717824 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.717786 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.718019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.717996 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmbsp\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-kube-api-access-fmbsp\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:09.828140 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:09.828057 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:10.060817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.060591 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:10.167019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.166986 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h"] Oct 30 20:02:10.167746 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:02:10.167718 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9738fae3_4f39_49cd_8368_02e1ea0134e9.slice/crio-b2f1b90375aec69886cf29eba504f699b196c7d6cd0dce500848ee8c8b765bd8 WatchSource:0}: Error finding container b2f1b90375aec69886cf29eba504f699b196c7d6cd0dce500848ee8c8b765bd8: Status 404 returned error can't find the container with id b2f1b90375aec69886cf29eba504f699b196c7d6cd0dce500848ee8c8b765bd8 Oct 30 20:02:10.170664 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.170629 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:10.170755 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.170711 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:10.170755 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.170738 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:10.378334 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.378295 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:10.567011 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.566916 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" event={"ID":"9738fae3-4f39-49cd-8368-02e1ea0134e9","Type":"ContainerStarted","Data":"a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02"} Oct 30 20:02:10.567011 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.567002 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" event={"ID":"9738fae3-4f39-49cd-8368-02e1ea0134e9","Type":"ContainerStarted","Data":"b2f1b90375aec69886cf29eba504f699b196c7d6cd0dce500848ee8c8b765bd8"} Oct 30 20:02:10.622440 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.622376 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" podStartSLOduration=1.62235876 podStartE2EDuration="1.62235876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:02:10.620226717 +0000 UTC m=+847.020398762" watchObservedRunningTime="2026-10-30 20:02:10.62235876 +0000 UTC m=+847.022530767" Oct 30 20:02:10.828319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:10.828234 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:11.830499 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:11.830464 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 20:02:11.830915 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:11.830525 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 20:02:12.401457 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:12.401422 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:12.828641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:12.828554 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 20:02:12.828641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:12.828610 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 20:02:13.828714 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:13.828675 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 20:02:13.829146 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:13.828738 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 20:02:14.368805 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:14.368726 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:14.840920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:14.840362 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 20:02:14.840920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:14.840426 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 20:02:15.302495 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:15.302460 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:15.829187 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:15.829144 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 20:02:15.829399 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:15.829209 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 20:02:16.134192 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:16.134145 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:17.161679 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:17.161649 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:17.162356 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:17.162318 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:17.163435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:17.163412 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:02:18.034109 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:18.034073 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:19.820435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:19.820404 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:21.637933 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:21.637838 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:23.457529 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:23.457450 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:24.362748 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.362717 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd"] Oct 30 20:02:24.367059 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.367031 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.392731 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.391410 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd"] Oct 30 20:02:24.428854 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.428823 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g"] Oct 30 20:02:24.437960 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.437936 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.443388 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.440410 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g"] Oct 30 20:02:24.463951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.463161 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-626870-openshift-default-dockercfg-8x78h\"" Oct 30 20:02:24.528220 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.528179 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87bef8f5-1dad-404f-9386-aed32dc162a8-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.528566 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.528543 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.528751 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.528736 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.528953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.528933 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcdws\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-kube-api-access-mcdws\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.529109 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.529094 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.529259 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.529242 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.529410 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.529395 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.529589 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.529574 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.529797 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.529781 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.529973 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.529958 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.530125 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.530111 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/130f8cd2-8f57-4058-8f1f-295c0718de86-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.530266 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.530252 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.530451 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.530435 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bmb5\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-kube-api-access-4bmb5\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.530595 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.530580 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.530754 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.530738 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.530946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.530880 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.531120 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.531098 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.531242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.531146 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632205 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632171 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.632379 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632230 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632379 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632264 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.632379 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632286 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/130f8cd2-8f57-4058-8f1f-295c0718de86-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632379 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632316 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632379 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632368 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bmb5\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-kube-api-access-4bmb5\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632394 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632533 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.632641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632609 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.632641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632640 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.632927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632683 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632771 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87bef8f5-1dad-404f-9386-aed32dc162a8-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.632927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632822 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632820 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632829 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.632927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.632842 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.633240 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633078 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/130f8cd2-8f57-4058-8f1f-295c0718de86-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.633240 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633156 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcdws\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-kube-api-access-mcdws\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.633240 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633189 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.633240 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633224 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.633433 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633256 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.633433 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633335 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.633550 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633514 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.633610 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633513 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-certs\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.633666 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633618 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-credential-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.633797 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633776 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-socket\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.633867 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.633836 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-data\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.634395 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.634354 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87bef8f5-1dad-404f-9386-aed32dc162a8-istiod-ca-cert\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.635254 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.635230 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-podinfo\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.635557 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.635534 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-envoy\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.635722 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.635702 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.635986 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.635967 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.660329 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.660303 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bmb5\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-kube-api-access-4bmb5\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.661261 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.661231 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-token\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.661447 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.661430 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:24.661524 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.661513 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcdws\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-kube-api-access-mcdws\") pod \"e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.684585 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.684561 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:24.761926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:24.761079 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:25.015637 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.015555 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd"] Oct 30 20:02:25.029071 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.029025 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:25.029196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.029123 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:25.029196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.029170 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:25.108187 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.108164 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g"] Oct 30 20:02:25.110501 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:02:25.110478 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130f8cd2_8f57_4058_8f1f_295c0718de86.slice/crio-30fa9e110314dcf1d7af6001787d5f6fd1a61a3976d69dc78488360fcd459250 WatchSource:0}: Error finding container 30fa9e110314dcf1d7af6001787d5f6fd1a61a3976d69dc78488360fcd459250: Status 404 returned error can't find the container with id 30fa9e110314dcf1d7af6001787d5f6fd1a61a3976d69dc78488360fcd459250 Oct 30 20:02:25.113049 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.113015 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:25.113137 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.113080 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:25.113137 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.113125 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:25.626961 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.626886 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:25.676383 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.676344 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" event={"ID":"130f8cd2-8f57-4058-8f1f-295c0718de86","Type":"ContainerStarted","Data":"f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132"} Oct 30 20:02:25.676570 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.676417 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" event={"ID":"130f8cd2-8f57-4058-8f1f-295c0718de86","Type":"ContainerStarted","Data":"30fa9e110314dcf1d7af6001787d5f6fd1a61a3976d69dc78488360fcd459250"} Oct 30 20:02:25.695930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.695291 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" event={"ID":"87bef8f5-1dad-404f-9386-aed32dc162a8","Type":"ContainerStarted","Data":"31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3"} Oct 30 20:02:25.695930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.695356 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" event={"ID":"87bef8f5-1dad-404f-9386-aed32dc162a8","Type":"ContainerStarted","Data":"4c0a70b5d56d1da0028bec9946bada23204b3953253d4ffc3c4b38042d2b465f"} Oct 30 20:02:25.732923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.731033 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podStartSLOduration=1.731016491 podStartE2EDuration="1.731016491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:02:25.729446682 +0000 UTC m=+862.129618693" watchObservedRunningTime="2026-10-30 20:02:25.731016491 +0000 UTC m=+862.131188499" Oct 30 20:02:25.762023 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:25.761991 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:26.701542 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:26.686352 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:26.741091 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:26.741048 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Oct 30 20:02:26.741260 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:26.741161 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Oct 30 20:02:26.764603 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:26.764574 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 20:02:26.764740 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:26.764626 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 20:02:27.631918 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:27.631717 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:27.705922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:27.704158 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Oct 30 20:02:27.705922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:27.704219 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Oct 30 20:02:27.769944 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:27.762454 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 20:02:27.769944 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:27.762518 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 20:02:28.695922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:28.686286 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Oct 30 20:02:28.695922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:28.686360 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Oct 30 20:02:28.762774 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:28.762723 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 20:02:28.763181 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:28.762849 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 20:02:29.701463 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:29.701004 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Oct 30 20:02:29.701463 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:29.701069 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Oct 30 20:02:29.736369 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:29.736332 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:29.762621 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:29.762589 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 20:02:29.762800 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:29.762651 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 20:02:30.687378 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:30.687340 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Oct 30 20:02:30.687686 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:30.687658 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Oct 30 20:02:30.762787 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:30.762757 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 20:02:30.763127 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:30.763103 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 20:02:31.685536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:31.685499 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Oct 30 20:02:31.685733 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:31.685575 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Oct 30 20:02:31.762394 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:31.762360 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 20:02:31.762558 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:31.762437 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 20:02:31.907092 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:31.907053 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:32.690543 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:32.690509 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:32.690955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:32.690928 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:32.691563 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:32.691545 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:02:32.752348 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:32.752305 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podStartSLOduration=8.752285054 podStartE2EDuration="8.752285054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:02:25.801114119 +0000 UTC m=+862.201286139" watchObservedRunningTime="2026-10-30 20:02:32.752285054 +0000 UTC m=+869.152457060" Oct 30 20:02:32.766740 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:32.766716 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:32.767005 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:32.766987 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:32.767555 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:32.767538 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:02:33.732435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:33.732403 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:34.449475 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:34.449429 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:35.149749 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:02:35.149706 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:02:35.929206 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:35.929166 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:37.717498 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:37.717455 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:39.702675 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:39.702621 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:40.126646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.126567 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf"] Oct 30 20:02:40.133007 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.132986 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.167570 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.167528 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-912f24-openshift-default-dockercfg-pzcfl\"" Oct 30 20:02:40.168744 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.168708 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf"] Oct 30 20:02:40.184044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184014 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.184308 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184268 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7k5x\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-kube-api-access-s7k5x\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.184423 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184401 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.184519 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184501 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8904408-b320-477b-9ab8-42498ca0a1c5-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.184594 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184570 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.184652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184600 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.184652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184629 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.184759 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184667 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.184759 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.184720 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.274419 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.274388 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl"] Oct 30 20:02:40.278696 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.278669 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.285677 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.285647 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.285829 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.285689 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7k5x\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-kube-api-access-s7k5x\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.285829 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.285770 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.285991 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.285859 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8904408-b320-477b-9ab8-42498ca0a1c5-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.285991 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.285945 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.285991 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.285976 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.286742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.286008 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.286742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.286044 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.286742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.286099 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.286742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.286106 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.286742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.286508 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.287694 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.287670 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.287846 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.287733 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8904408-b320-477b-9ab8-42498ca0a1c5-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.288035 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.287975 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.289170 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.289148 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.290149 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.290130 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.299312 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.299292 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-51dd62-openshift-default-dockercfg-678jt\"" Oct 30 20:02:40.311203 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.310687 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl"] Oct 30 20:02:40.321621 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.321584 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.321845 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.321822 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7k5x\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-kube-api-access-s7k5x\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.387826 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.387782 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.388041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.387965 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.388041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.388010 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.388141 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.388094 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljslf\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-kube-api-access-ljslf\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.388223 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.388207 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.388267 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.388256 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.388327 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.388301 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.388461 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.388437 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9733f9d7-b928-4bd7-be04-9ac327153c27-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.388509 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.388493 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.446792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.446763 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:40.489273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489243 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.489444 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489339 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.489444 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489419 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.489551 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489452 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.489551 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489486 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljslf\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-kube-api-access-ljslf\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.489663 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489576 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.489663 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489626 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.489765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489664 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.489765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.489715 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9733f9d7-b928-4bd7-be04-9ac327153c27-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.490126 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.490104 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.490386 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.490364 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.490489 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.490471 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.490657 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.490633 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.491344 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.491255 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9733f9d7-b928-4bd7-be04-9ac327153c27-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.495085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.493066 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.495085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.493888 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.516671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.516118 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljslf\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-kube-api-access-ljslf\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.530735 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.517650 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.595991 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.595942 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:40.780563 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.780534 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf"] Oct 30 20:02:40.782420 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:02:40.782394 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8904408_b320_477b_9ab8_42498ca0a1c5.slice/crio-7da4604130c0a1cdd2a2a17c102451b4039b602baa22791706b1140efd1c2962 WatchSource:0}: Error finding container 7da4604130c0a1cdd2a2a17c102451b4039b602baa22791706b1140efd1c2962: Status 404 returned error can't find the container with id 7da4604130c0a1cdd2a2a17c102451b4039b602baa22791706b1140efd1c2962 Oct 30 20:02:40.785299 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.785258 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:40.785383 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.785344 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:40.785420 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.785386 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:40.909569 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.909486 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl"] Oct 30 20:02:40.915399 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:02:40.915367 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9733f9d7_b928_4bd7_be04_9ac327153c27.slice/crio-9e1979bb97529da97a7088754254c24f81ceb4c3e794eb01b5d02a5a1ca896f9 WatchSource:0}: Error finding container 9e1979bb97529da97a7088754254c24f81ceb4c3e794eb01b5d02a5a1ca896f9: Status 404 returned error can't find the container with id 9e1979bb97529da97a7088754254c24f81ceb4c3e794eb01b5d02a5a1ca896f9 Oct 30 20:02:40.918194 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.918155 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:40.918317 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.918242 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:40.918317 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:40.918279 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:02:41.022105 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.021084 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" event={"ID":"9733f9d7-b928-4bd7-be04-9ac327153c27","Type":"ContainerStarted","Data":"a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e"} Oct 30 20:02:41.022105 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.021149 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" event={"ID":"9733f9d7-b928-4bd7-be04-9ac327153c27","Type":"ContainerStarted","Data":"9e1979bb97529da97a7088754254c24f81ceb4c3e794eb01b5d02a5a1ca896f9"} Oct 30 20:02:41.024297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.024264 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" event={"ID":"b8904408-b320-477b-9ab8-42498ca0a1c5","Type":"ContainerStarted","Data":"c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f"} Oct 30 20:02:41.024705 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.024682 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" event={"ID":"b8904408-b320-477b-9ab8-42498ca0a1c5","Type":"ContainerStarted","Data":"7da4604130c0a1cdd2a2a17c102451b4039b602baa22791706b1140efd1c2962"} Oct 30 20:02:41.073923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.070517 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podStartSLOduration=1.070498684 podStartE2EDuration="1.070498684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:02:41.069518628 +0000 UTC m=+877.469690636" watchObservedRunningTime="2026-10-30 20:02:41.070498684 +0000 UTC m=+877.470670696" Oct 30 20:02:41.120044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.119570 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podStartSLOduration=1.119554905 podStartE2EDuration="1.119554905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:02:41.115709062 +0000 UTC m=+877.515881070" watchObservedRunningTime="2026-10-30 20:02:41.119554905 +0000 UTC m=+877.519726911" Oct 30 20:02:41.447719 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.447679 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:41.448936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.448914 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 20:02:41.449056 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.448965 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 20:02:41.597735 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.596993 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:41.605772 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:41.605709 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:42.447921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:42.447787 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 20:02:42.447921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:42.447838 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 20:02:42.613925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:42.605068 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 20:02:42.613925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:42.605132 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 20:02:43.449256 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:43.449218 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 20:02:43.449840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:43.449811 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 20:02:43.503945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:43.503887 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:43.599044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:43.599011 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 20:02:43.599324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:43.599299 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 20:02:44.452677 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:44.452630 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 20:02:44.453210 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:44.452704 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 20:02:44.605950 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:44.597792 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 20:02:44.605950 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:44.597869 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 20:02:45.329495 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:45.329460 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:45.460437 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:45.460345 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 20:02:45.460437 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:45.460410 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 20:02:45.600880 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:45.600344 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 20:02:45.600880 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:45.600429 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 20:02:46.449329 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:46.449290 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 20:02:46.449620 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:46.449592 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 20:02:46.608922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:46.608341 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 20:02:46.608922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:46.608423 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 20:02:47.246923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:47.238652 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:47.448567 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:47.448527 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 20:02:47.448843 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:47.448811 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 20:02:47.597406 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:47.597313 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 20:02:47.597406 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:47.597379 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 20:02:49.150206 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:49.149290 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:49.150206 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:49.149773 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:49.151567 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:49.151501 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:02:49.156690 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:49.156654 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:49.404052 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:49.403960 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:49.404816 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:49.404797 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:49.406455 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:49.406434 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:02:50.936128 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:50.936095 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:52.740663 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:52.740594 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:54.286925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:54.285245 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:02:54.566983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:54.565018 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:02:54.895669 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:54.895624 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:55.362986 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:55.361989 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:02:56.733819 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:56.733570 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:02:58.728382 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:02:58.728343 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:00.794884 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:00.794843 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:02.597127 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:02.597098 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:04.425854 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:04.425818 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:05.097945 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:03:05.097887 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:03:05.497813 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:05.497768 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:06.305528 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:06.305482 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:08.206632 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:08.206590 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:09.638927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.614722 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx"] Oct 30 20:03:09.719921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.719005 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.917986 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.916926 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.925130 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.925093 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.925433 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.925393 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.925595 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.925578 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.925785 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.925751 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv9gl\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-kube-api-access-vv9gl\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.933199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.933172 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.933501 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.933463 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.948271 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.948226 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4f744d-54e4-47d1-944b-419a9d845c8b-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.948538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.948502 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:09.973957 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:09.973231 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx"] Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.061638 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4f744d-54e4-47d1-944b-419a9d845c8b-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.061723 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.061769 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.061847 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.061885 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.061935 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.061987 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv9gl\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-kube-api-access-vv9gl\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.062053 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.064936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.062127 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.125943 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.066260 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.126243 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.126197 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.126365 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.070624 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4f744d-54e4-47d1-944b-419a9d845c8b-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.126365 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.070864 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.126365 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.071390 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.126365 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.097677 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.126668 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.069215 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.155969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.148604 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv9gl\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-kube-api-access-vv9gl\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.249654 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.249573 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.281802 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.281698 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:10.361047 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.361000 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:03:10.597010 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.575205 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:10.799370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:10.799332 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:11.253807 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:11.253521 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:03:11.408254 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:11.377122 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:03:11.408254 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:11.377624 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:03:11.420466 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:11.419865 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:03:11.665844 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:11.665749 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx"] Oct 30 20:03:11.698635 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:11.698592 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:03:11.698854 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:11.698830 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:03:11.699028 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:11.699004 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:03:12.686191 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:12.686148 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" event={"ID":"4c4f744d-54e4-47d1-944b-419a9d845c8b","Type":"ContainerStarted","Data":"d1d48d91f9feac5515189b72f794deed33786fb80e12767afdc701f4fa62916d"} Oct 30 20:03:12.686736 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:12.686223 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" event={"ID":"4c4f744d-54e4-47d1-944b-419a9d845c8b","Type":"ContainerStarted","Data":"f1323a987efb1771b2fba5f0c775a38dcef0254399addc4408980befcee8f850"} Oct 30 20:03:12.749974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:12.749922 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podStartSLOduration=3.749891242 podStartE2EDuration="3.749891242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:12.748839933 +0000 UTC m=+909.149011951" watchObservedRunningTime="2026-10-30 20:03:12.749891242 +0000 UTC m=+909.150063249" Oct 30 20:03:12.869556 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:12.869501 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:13.292287 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:13.292249 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:13.308044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:13.308005 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 20:03:13.308258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:13.308070 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 20:03:14.290923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:14.286223 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 20:03:14.290923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:14.286282 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 20:03:15.112701 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:15.112669 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:15.282229 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:15.282196 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 20:03:15.282498 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:15.282471 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 20:03:16.285223 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:16.285190 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 20:03:16.285812 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:16.285780 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 20:03:17.145022 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:17.144968 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:17.303924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:17.284220 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 20:03:17.303924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:17.284299 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 20:03:18.283045 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:18.283007 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 20:03:18.283251 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:18.283071 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 20:03:19.112185 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:19.112152 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:19.286766 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:19.286735 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:19.287153 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:19.287130 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:19.287851 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:19.287835 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:03:20.898655 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:20.898616 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:22.644175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:22.644144 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:24.758347 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:24.757096 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:26.528257 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:26.528220 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:28.429328 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:28.429226 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:29.337446 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.337396 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r"] Oct 30 20:03:29.342211 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.342184 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.371106 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.371029 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-62f4a1\"/\"distinct-w0-62f4a1-kserve-self-signed-certs\"" Oct 30 20:03:29.374743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.374716 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r"] Oct 30 20:03:29.387319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.387292 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:29.419284 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.419256 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phrl6\" (UniqueName: \"kubernetes.io/projected/981c36db-e723-499d-802f-cbfb370addee-kube-api-access-phrl6\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.419416 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.419394 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-home\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.419608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.419582 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-dshm\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.419670 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.419652 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-model-cache\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.419730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.419679 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981c36db-e723-499d-802f-cbfb370addee-tls-certs\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.419783 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.419750 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-tmp-dir\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.521391 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.521359 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-home\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.521875 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.521459 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-dshm\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.521875 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.521487 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-model-cache\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.521875 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.521504 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981c36db-e723-499d-802f-cbfb370addee-tls-certs\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.521875 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.521550 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-tmp-dir\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.521875 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.521576 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phrl6\" (UniqueName: \"kubernetes.io/projected/981c36db-e723-499d-802f-cbfb370addee-kube-api-access-phrl6\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.522248 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.522228 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-home\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.523191 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.523159 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-model-cache\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.523658 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.523540 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-tmp-dir\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.525379 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.525353 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-dshm\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.525980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.525942 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981c36db-e723-499d-802f-cbfb370addee-tls-certs\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.549863 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.549829 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phrl6\" (UniqueName: \"kubernetes.io/projected/981c36db-e723-499d-802f-cbfb370addee-kube-api-access-phrl6\") pod \"distinct-w0-62f4a1-kserve-7646b99569-7cq8r\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.656421 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.656385 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:29.964378 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:29.964352 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r"] Oct 30 20:03:29.966952 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:03:29.966918 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod981c36db_e723_499d_802f_cbfb370addee.slice/crio-3d34b86b4e2433a6359ff59e03bd8fd527ff6a0828de235cc8a008c44008e89f WatchSource:0}: Error finding container 3d34b86b4e2433a6359ff59e03bd8fd527ff6a0828de235cc8a008c44008e89f: Status 404 returned error can't find the container with id 3d34b86b4e2433a6359ff59e03bd8fd527ff6a0828de235cc8a008c44008e89f Oct 30 20:03:30.397428 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:30.397393 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:30.854786 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:30.854703 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" event={"ID":"981c36db-e723-499d-802f-cbfb370addee","Type":"ContainerStarted","Data":"7ca45219e56ddcd5ab7bebecaad1f78624e2bf346e046ca2a14c258e49436792"} Oct 30 20:03:30.854786 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:30.854753 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" event={"ID":"981c36db-e723-499d-802f-cbfb370addee","Type":"ContainerStarted","Data":"3d34b86b4e2433a6359ff59e03bd8fd527ff6a0828de235cc8a008c44008e89f"} Oct 30 20:03:30.902136 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:30.902088 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podStartSLOduration=1.9020749270000001 podStartE2EDuration="1.902074927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:30.899766459 +0000 UTC m=+927.299938467" watchObservedRunningTime="2026-10-30 20:03:30.902074927 +0000 UTC m=+927.302246933" Oct 30 20:03:31.656919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:31.656865 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:32.259511 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:32.259468 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:34.214116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:34.214075 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:35.096606 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:03:35.096571 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:03:35.920422 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:35.920385 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:37.932966 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:37.932921 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:39.657337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.657293 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:39.670726 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.670696 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:39.737403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.737370 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht"] Oct 30 20:03:39.741593 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.741571 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.759652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.759625 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-912f24\"/\"kube-root-ca.crt\"" Oct 30 20:03:39.759810 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.759672 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-912f24\"/\"default-dockercfg-hw2ql\"" Oct 30 20:03:39.759810 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.759791 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-912f24\"/\"distinct-w3-912f24-kserve-self-signed-certs\"" Oct 30 20:03:39.761057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.761032 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-912f24\"/\"openshift-service-ca.crt\"" Oct 30 20:03:39.762733 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.762709 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:39.763158 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.763140 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht"] Oct 30 20:03:39.763496 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.763475 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93138c42-74ff-4d13-b362-4f859beb67df-tls-certs\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.763642 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.763626 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-model-cache\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.763688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.763670 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-dshm\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.763735 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.763721 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-tmp-dir\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.763951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.763933 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v72xb\" (UniqueName: \"kubernetes.io/projected/93138c42-74ff-4d13-b362-4f859beb67df-kube-api-access-v72xb\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.764039 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.764025 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-home\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865113 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865084 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v72xb\" (UniqueName: \"kubernetes.io/projected/93138c42-74ff-4d13-b362-4f859beb67df-kube-api-access-v72xb\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865328 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865148 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-home\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865328 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865183 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93138c42-74ff-4d13-b362-4f859beb67df-tls-certs\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865328 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865256 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-model-cache\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865328 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865287 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-dshm\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865328 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865313 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-tmp-dir\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865632 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865561 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-home\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865739 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865706 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-tmp-dir\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.865925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.865887 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-model-cache\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.867632 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.867605 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-dshm\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.867958 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.867942 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93138c42-74ff-4d13-b362-4f859beb67df-tls-certs\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.891482 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.891461 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v72xb\" (UniqueName: \"kubernetes.io/projected/93138c42-74ff-4d13-b362-4f859beb67df-kube-api-access-v72xb\") pod \"distinct-w3-912f24-kserve-7f44b47cc8-4sjht\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:39.944629 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:39.944533 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:03:40.007295 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:40.007242 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:40.055360 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:40.055327 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:40.366272 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:40.366245 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht"] Oct 30 20:03:40.368068 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:03:40.368046 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93138c42_74ff_4d13_b362_4f859beb67df.slice/crio-b178e37eb6301122f35891e6193d9f177e11e6aac6b3d63c56bc5d7f9b1c1382 WatchSource:0}: Error finding container b178e37eb6301122f35891e6193d9f177e11e6aac6b3d63c56bc5d7f9b1c1382: Status 404 returned error can't find the container with id b178e37eb6301122f35891e6193d9f177e11e6aac6b3d63c56bc5d7f9b1c1382 Oct 30 20:03:40.924120 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:40.924080 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" event={"ID":"93138c42-74ff-4d13-b362-4f859beb67df","Type":"ContainerStarted","Data":"bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185"} Oct 30 20:03:40.924635 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:40.924137 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" event={"ID":"93138c42-74ff-4d13-b362-4f859beb67df","Type":"ContainerStarted","Data":"b178e37eb6301122f35891e6193d9f177e11e6aac6b3d63c56bc5d7f9b1c1382"} Oct 30 20:03:40.969437 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:40.968942 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podStartSLOduration=1.968925197 podStartE2EDuration="1.968925197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:40.968004448 +0000 UTC m=+937.368176455" watchObservedRunningTime="2026-10-30 20:03:40.968925197 +0000 UTC m=+937.369097206" Oct 30 20:03:41.056345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:41.056310 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:41.706809 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:41.706774 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:43.507926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:43.507877 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:45.333140 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:45.333108 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:47.261867 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:47.261828 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:49.119577 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:49.119539 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:50.055993 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:50.055961 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:50.068380 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:50.068354 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:50.991264 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:50.991226 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:50.997110 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:50.997087 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:03:52.506830 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:52.506799 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:52.785680 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:52.785589 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:54.601189 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:54.601160 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:56.517538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:56.517505 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:03:58.302913 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:03:58.302875 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:00.236509 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:00.236472 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:02.219662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:02.219596 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:03.884490 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.884458 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl"] Oct 30 20:04:03.888689 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.888669 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:03.910106 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.910080 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl"] Oct 30 20:04:03.910711 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.910687 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:03.919971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.919949 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-62f4a1\"/\"distinct-2-w0-62f4a1-kserve-self-signed-certs\"" Oct 30 20:04:03.971738 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.971709 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-tmp-dir\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:03.971909 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.971835 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f5fddd-2204-412a-bee6-ddd2dbe13213-tls-certs\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:03.971975 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.971885 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbw6x\" (UniqueName: \"kubernetes.io/projected/00f5fddd-2204-412a-bee6-ddd2dbe13213-kube-api-access-wbw6x\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:03.972218 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.972197 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-model-cache\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:03.972432 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.972408 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-dshm\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:03.972644 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:03.972602 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-home\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.074378 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074343 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-tmp-dir\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.074571 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074398 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f5fddd-2204-412a-bee6-ddd2dbe13213-tls-certs\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.074571 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074418 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbw6x\" (UniqueName: \"kubernetes.io/projected/00f5fddd-2204-412a-bee6-ddd2dbe13213-kube-api-access-wbw6x\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.074571 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074485 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-model-cache\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.074571 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074563 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-dshm\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.074852 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074621 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-home\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.074852 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074799 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-tmp-dir\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.075120 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074874 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-model-cache\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.075120 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.074993 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-home\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.077427 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.077402 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-dshm\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.078278 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.078248 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f5fddd-2204-412a-bee6-ddd2dbe13213-tls-certs\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.106938 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.106887 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbw6x\" (UniqueName: \"kubernetes.io/projected/00f5fddd-2204-412a-bee6-ddd2dbe13213-kube-api-access-wbw6x\") pod \"distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.145460 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.145338 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:04.203810 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.203778 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:04.534854 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:04.534773 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl"] Oct 30 20:04:04.537862 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:04:04.537839 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00f5fddd_2204_412a_bee6_ddd2dbe13213.slice/crio-c879b88f2aa600096c27e4975a38d8b4b208efe5ddd4e84b766ef3a894eedb1a WatchSource:0}: Error finding container c879b88f2aa600096c27e4975a38d8b4b208efe5ddd4e84b766ef3a894eedb1a: Status 404 returned error can't find the container with id c879b88f2aa600096c27e4975a38d8b4b208efe5ddd4e84b766ef3a894eedb1a Oct 30 20:04:05.083398 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:05.083365 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" event={"ID":"00f5fddd-2204-412a-bee6-ddd2dbe13213","Type":"ContainerStarted","Data":"dd8f3397940dffb3b0a7e42f0c63788cc82a3cf9dbdd9f47c2d980f42d30ab9b"} Oct 30 20:04:05.083832 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:05.083413 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" event={"ID":"00f5fddd-2204-412a-bee6-ddd2dbe13213","Type":"ContainerStarted","Data":"c879b88f2aa600096c27e4975a38d8b4b208efe5ddd4e84b766ef3a894eedb1a"} Oct 30 20:04:05.094845 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:04:05.094811 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:04:05.130034 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:05.129989 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podStartSLOduration=2.129972581 podStartE2EDuration="2.129972581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:05.129637175 +0000 UTC m=+961.529809182" watchObservedRunningTime="2026-10-30 20:04:05.129972581 +0000 UTC m=+961.530144589" Oct 30 20:04:05.204318 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:05.204263 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:05.951920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:05.951845 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:07.744255 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:07.744227 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:09.548123 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:09.548087 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:10.429296 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.429260 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd"] Oct 30 20:04:10.433613 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.433592 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.466257 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.466226 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd"] Oct 30 20:04:10.552121 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552081 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.552121 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552127 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.552532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552215 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.552532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552282 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.552532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552354 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.552532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552409 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.552532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552487 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-468lt\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-kube-api-access-468lt\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.552532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552509 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.552532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.552531 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.636528 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.636497 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:10.653352 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653323 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653467 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653385 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-468lt\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-kube-api-access-468lt\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653467 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653415 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653467 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653435 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653575 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653504 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653575 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653532 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653673 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653584 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653673 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653650 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653778 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653685 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.653933 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653885 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.654016 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.653950 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.654146 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.654121 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.654270 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.654210 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.654465 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.654446 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.655736 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.655718 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.655978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.655962 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.679050 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.679024 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.679185 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.679162 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-468lt\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-kube-api-access-468lt\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:10.747546 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:10.747481 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:11.073167 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:11.073139 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd"] Oct 30 20:04:11.075571 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:04:11.075544 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aac11e2_19d4_490f_a67f_af5e5ad9ba1c.slice/crio-08b10df81267ace365159745798294f4412f1a51888c5d4cdccdcd822a6e1784 WatchSource:0}: Error finding container 08b10df81267ace365159745798294f4412f1a51888c5d4cdccdcd822a6e1784: Status 404 returned error can't find the container with id 08b10df81267ace365159745798294f4412f1a51888c5d4cdccdcd822a6e1784 Oct 30 20:04:11.077925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:11.077874 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:11.078018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:11.077972 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:11.078058 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:11.078018 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:11.125947 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:11.125914 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" event={"ID":"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c","Type":"ContainerStarted","Data":"08b10df81267ace365159745798294f4412f1a51888c5d4cdccdcd822a6e1784"} Oct 30 20:04:11.427677 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:11.427633 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:12.137753 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:12.136357 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" event={"ID":"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c","Type":"ContainerStarted","Data":"7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9"} Oct 30 20:04:12.206574 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:12.206512 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podStartSLOduration=2.206492912 podStartE2EDuration="2.206492912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:12.202517296 +0000 UTC m=+968.602689305" watchObservedRunningTime="2026-10-30 20:04:12.206492912 +0000 UTC m=+968.606664920" Oct 30 20:04:12.748089 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:12.748058 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:12.749579 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:12.749553 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 20:04:12.749704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:12.749601 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 20:04:13.324115 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:13.324077 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:13.748541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:13.748506 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 20:04:13.748772 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:13.748560 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 20:04:14.204954 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:14.204879 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:14.218625 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:14.218599 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:14.748819 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:14.748777 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 20:04:14.749261 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:14.748860 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 20:04:15.167679 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:15.167650 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:15.246086 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:15.242984 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:15.748276 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:15.748241 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 20:04:15.748450 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:15.748313 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 20:04:16.747813 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:16.747778 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 20:04:16.748224 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:16.747841 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 20:04:17.230297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:17.230264 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:17.748692 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:17.748662 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 20:04:17.749091 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:17.748710 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 20:04:18.752714 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:18.752684 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:18.753168 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:18.753004 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:18.753889 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:18.753873 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:04:19.050706 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:19.050629 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:20.802517 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:20.802485 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:21.519855 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:21.519821 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:22.716088 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:22.716051 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:24.539369 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:24.539335 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:25.211841 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.211806 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5"] Oct 30 20:04:25.216273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.216246 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.247647 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.247610 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5"] Oct 30 20:04:25.403230 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403196 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/860bfb55-555b-4198-b9ae-23712a7453e1-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.403431 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403246 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.403431 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403270 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/860bfb55-555b-4198-b9ae-23712a7453e1-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.403431 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403421 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.403585 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403496 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.403681 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403652 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.403775 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403701 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84b85\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-kube-api-access-84b85\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.403846 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403817 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.403889 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.403871 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505001 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.504879 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505001 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.504954 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84b85\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-kube-api-access-84b85\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505005 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505043 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505090 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/860bfb55-555b-4198-b9ae-23712a7453e1-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505120 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505144 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/860bfb55-555b-4198-b9ae-23712a7453e1-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505479 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505238 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505479 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505311 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505479 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505357 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505479 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505435 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505733 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505711 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505803 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505782 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.505851 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.505833 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/860bfb55-555b-4198-b9ae-23712a7453e1-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.507426 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.507406 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.507837 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.507822 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/860bfb55-555b-4198-b9ae-23712a7453e1-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.532411 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.532383 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.532544 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.532415 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84b85\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-kube-api-access-84b85\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:25.761263 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.761147 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:25.832161 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:25.832128 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:26.162967 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:04:26.162937 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860bfb55_555b_4198_b9ae_23712a7453e1.slice/crio-6c8c8175a04d899cbe09cbb4f1d28dbc7ca1c897d08b7814b4398d0853b87122 WatchSource:0}: Error finding container 6c8c8175a04d899cbe09cbb4f1d28dbc7ca1c897d08b7814b4398d0853b87122: Status 404 returned error can't find the container with id 6c8c8175a04d899cbe09cbb4f1d28dbc7ca1c897d08b7814b4398d0853b87122 Oct 30 20:04:26.166947 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:26.166334 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:26.166947 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:26.166439 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:26.166947 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:26.166503 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:26.167427 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:26.167405 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5"] Oct 30 20:04:26.233835 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:26.233811 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" event={"ID":"860bfb55-555b-4198-b9ae-23712a7453e1","Type":"ContainerStarted","Data":"6c8c8175a04d899cbe09cbb4f1d28dbc7ca1c897d08b7814b4398d0853b87122"} Oct 30 20:04:26.819165 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:26.819123 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:27.241361 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:27.241315 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" event={"ID":"860bfb55-555b-4198-b9ae-23712a7453e1","Type":"ContainerStarted","Data":"3f412cd1a29e5cdaf2eff2c4411aed99a1c22f40f3234ea844f0de5d8e4d6de4"} Oct 30 20:04:27.832323 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:27.832295 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:27.833987 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:27.833966 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Oct 30 20:04:27.834037 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:27.834012 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Oct 30 20:04:28.832933 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:28.832867 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Oct 30 20:04:28.833418 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:28.832961 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Oct 30 20:04:28.999171 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:28.999118 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:29.832999 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:29.832955 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Oct 30 20:04:29.833495 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:29.833008 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Oct 30 20:04:30.832913 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:30.832843 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Oct 30 20:04:30.833117 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:30.832964 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Oct 30 20:04:31.151373 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:31.151337 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:31.833067 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:31.833027 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Oct 30 20:04:31.833246 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:31.833089 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Oct 30 20:04:32.839287 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:32.839250 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Oct 30 20:04:32.839920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:32.839871 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Oct 30 20:04:33.020656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:33.020617 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:33.837546 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:33.837512 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:33.838411 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:33.838370 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:33.839446 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:33.839420 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:04:33.905916 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:33.905849 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podStartSLOduration=8.905835295 podStartE2EDuration="8.905835295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:27.311287112 +0000 UTC m=+983.711459119" watchObservedRunningTime="2026-10-30 20:04:33.905835295 +0000 UTC m=+990.306007302" Oct 30 20:04:35.121162 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:04:35.121125 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:04:35.255305 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.255121 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:35.275122 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.262591 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb"] Oct 30 20:04:35.325922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.324163 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb"] Oct 30 20:04:35.325922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.324263 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.376241 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.376171 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-626870\"/\"default-dockercfg-b6x4h\"" Oct 30 20:04:35.376241 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.376187 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-626870\"/\"kube-root-ca.crt\"" Oct 30 20:04:35.376241 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.376196 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-626870\"/\"openshift-service-ca.crt\"" Oct 30 20:04:35.446380 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.446348 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-626870\"/\"distinct-w5-626870-kserve-self-signed-certs\"" Oct 30 20:04:35.466796 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.466764 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-dshm\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.467015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.466877 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-model-cache\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.467195 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.467171 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940223-245c-4132-880c-dd2f75dfe9cf-tls-certs\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.467276 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.467213 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74nvh\" (UniqueName: \"kubernetes.io/projected/4f940223-245c-4132-880c-dd2f75dfe9cf-kube-api-access-74nvh\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.467349 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.467334 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-home\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.467405 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.467368 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-tmp-dir\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.510352 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.510209 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:35.568865 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.568818 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-model-cache\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.569078 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.568996 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940223-245c-4132-880c-dd2f75dfe9cf-tls-certs\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.569078 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.569028 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74nvh\" (UniqueName: \"kubernetes.io/projected/4f940223-245c-4132-880c-dd2f75dfe9cf-kube-api-access-74nvh\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.569196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.569118 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-home\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.569196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.569157 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-tmp-dir\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.569293 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.569231 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-dshm\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.569758 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.569729 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-home\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.569986 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.569962 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-model-cache\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.570073 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.570008 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-tmp-dir\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.572178 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.572158 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-dshm\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.573619 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.573598 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940223-245c-4132-880c-dd2f75dfe9cf-tls-certs\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.613150 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.613085 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74nvh\" (UniqueName: \"kubernetes.io/projected/4f940223-245c-4132-880c-dd2f75dfe9cf-kube-api-access-74nvh\") pod \"distinct-w5-626870-kserve-85b5597ddf-w8mqb\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:35.668145 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:35.668109 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:36.067146 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:36.067117 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb"] Oct 30 20:04:36.069072 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:04:36.069049 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f940223_245c_4132_880c_dd2f75dfe9cf.slice/crio-62ac115ac0ca459bdadeddf629700e3e6b9dfe01a86a28e3b2f0bc561b47ab60 WatchSource:0}: Error finding container 62ac115ac0ca459bdadeddf629700e3e6b9dfe01a86a28e3b2f0bc561b47ab60: Status 404 returned error can't find the container with id 62ac115ac0ca459bdadeddf629700e3e6b9dfe01a86a28e3b2f0bc561b47ab60 Oct 30 20:04:36.322993 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:36.322874 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" event={"ID":"4f940223-245c-4132-880c-dd2f75dfe9cf","Type":"ContainerStarted","Data":"9d53bd631d2dc7d744fa5fa9e7a035691ac46c4e95b8d708508f98f8d1a5928e"} Oct 30 20:04:36.322993 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:36.322952 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" event={"ID":"4f940223-245c-4132-880c-dd2f75dfe9cf","Type":"ContainerStarted","Data":"62ac115ac0ca459bdadeddf629700e3e6b9dfe01a86a28e3b2f0bc561b47ab60"} Oct 30 20:04:36.371356 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:36.371283 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podStartSLOduration=1.371261916 podStartE2EDuration="1.371261916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:36.367846475 +0000 UTC m=+992.768018483" watchObservedRunningTime="2026-10-30 20:04:36.371261916 +0000 UTC m=+992.771433924" Oct 30 20:04:36.669235 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:36.669202 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:37.136319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:37.136288 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:38.925353 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:38.925319 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:40.440684 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.440642 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj"] Oct 30 20:04:40.445093 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.445070 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.480715 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.480681 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj"] Oct 30 20:04:40.532256 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532229 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.532415 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532294 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.532465 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532445 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7401270-59d3-4f59-af0c-c7943dde243b-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.532502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532483 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.532539 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532500 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.532609 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532594 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.532695 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532683 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.532729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532703 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7401270-59d3-4f59-af0c-c7943dde243b-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.532729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.532722 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdk4x\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-kube-api-access-hdk4x\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634354 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634322 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634354 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634356 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7401270-59d3-4f59-af0c-c7943dde243b-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634597 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634375 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdk4x\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-kube-api-access-hdk4x\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634597 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634472 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634597 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634537 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634614 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7401270-59d3-4f59-af0c-c7943dde243b-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634654 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634695 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.634765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.634741 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.635141 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.635108 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.635141 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.635135 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.635337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.635197 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.635337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.635246 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.635614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.635382 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7401270-59d3-4f59-af0c-c7943dde243b-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.637112 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.637086 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.637230 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.637208 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7401270-59d3-4f59-af0c-c7943dde243b-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.660684 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.660650 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.662054 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.662035 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdk4x\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-kube-api-access-hdk4x\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.758644 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.758573 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:40.815791 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:40.815211 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:41.080134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:41.080109 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj"] Oct 30 20:04:41.081662 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:04:41.081638 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7401270_59d3_4f59_af0c_c7943dde243b.slice/crio-37627afce21a926d7964686c2bf264fc6ac4b7cd34204716cdeddce8d9340d5a WatchSource:0}: Error finding container 37627afce21a926d7964686c2bf264fc6ac4b7cd34204716cdeddce8d9340d5a: Status 404 returned error can't find the container with id 37627afce21a926d7964686c2bf264fc6ac4b7cd34204716cdeddce8d9340d5a Oct 30 20:04:41.084618 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:41.084580 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:41.084701 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:41.084650 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:41.084701 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:41.084682 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:41.356116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:41.356019 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" event={"ID":"c7401270-59d3-4f59-af0c-c7943dde243b","Type":"ContainerStarted","Data":"27b2c4d5c96c478e2ecf20505d4884fd0acf89a05de4e71120297257b6ac261d"} Oct 30 20:04:41.356116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:41.356077 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" event={"ID":"c7401270-59d3-4f59-af0c-c7943dde243b","Type":"ContainerStarted","Data":"37627afce21a926d7964686c2bf264fc6ac4b7cd34204716cdeddce8d9340d5a"} Oct 30 20:04:41.412167 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:41.412068 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podStartSLOduration=1.412052395 podStartE2EDuration="1.412052395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:41.409814191 +0000 UTC m=+997.809986219" watchObservedRunningTime="2026-10-30 20:04:41.412052395 +0000 UTC m=+997.812224401" Oct 30 20:04:41.759176 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:41.759147 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:42.645143 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:42.645087 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:42.760362 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:42.760331 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 20:04:42.760740 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:42.760385 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 20:04:43.759788 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:43.759751 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 20:04:43.760046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:43.759818 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 20:04:44.420164 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:44.420117 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:44.759781 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:44.759709 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 20:04:44.759781 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:44.759758 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 20:04:45.669223 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:45.669190 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:45.710409 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:45.710375 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:45.759035 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:45.759003 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 20:04:45.759186 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:45.759076 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 20:04:46.336526 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:46.336491 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:46.410263 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:46.410188 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:46.765925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:46.761346 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 20:04:46.765925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:46.761402 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 20:04:47.806955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:47.774134 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 20:04:47.806955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:47.774211 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 20:04:47.994213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:47.992719 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:04:48.165066 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.165022 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:04:48.258924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.241015 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:48.398924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.398008 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:48.446132 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.442999 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:04:48.572334 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.572263 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:48.755401 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.754999 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:04:48.755401 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.755067 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:48.762925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.761186 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 20:04:48.762925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.761236 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 20:04:48.921924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.921020 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:48.996083 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.996032 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:48.996956 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:48.996931 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:04:49.221942 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.190318 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:49.221942 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.190980 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:49.258931 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.248989 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:49.258931 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.249715 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:04:49.407509 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.407463 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:49.408558 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.408538 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:49.449933 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.444122 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:49.449933 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.444528 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:04:49.648978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.579785 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:04:49.648978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.580577 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:04:49.765525 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.765489 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 20:04:49.824375 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.824334 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 20:04:49.907968 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.907836 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:49.939926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.927993 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.083171 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:49.989538 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:50.083171 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.001990 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.219005 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.202111 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.247328 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.222025 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:04:50.281926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.252126 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.410932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.404271 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:04:50.423930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.411189 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.450934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.424487 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.476302 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.451381 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.483925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.452675 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:04:50.483925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.480052 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.634923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.634310 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.692945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.669020 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:50.976256 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:50.976215 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:04:51.032920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.031762 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:04:51.056587 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.056361 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:51.061558 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.061406 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:51.248077 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.247984 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:51.253914 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.253870 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:51.255142 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.255107 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:04:51.287301 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.287269 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:51.456185 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.456150 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:04:51.467233 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.467203 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:04:51.485455 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.484723 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:04:51.518410 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.518336 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:04:51.520630 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.520608 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:04:51.653516 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.653490 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:51.762660 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.762616 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:04:51.762833 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:51.762690 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:52.804102 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:52.804063 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:04:52.804729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:52.804698 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:04:53.206315 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:53.206285 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:53.206960 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:53.206935 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:53.207869 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:53.207845 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:04:53.705614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:53.705577 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:55.519673 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.519638 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:55.626446 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.626410 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv"] Oct 30 20:04:55.631370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.631349 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.656626 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.656597 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv"] Oct 30 20:04:55.752423 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752388 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.752592 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752429 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.752592 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752531 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.752668 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752600 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.752668 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752620 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.752668 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752656 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc6sk\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-kube-api-access-mc6sk\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.752762 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752673 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.752762 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752691 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.752973 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.752929 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854060 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.853992 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854060 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854024 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854060 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854047 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc6sk\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-kube-api-access-mc6sk\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854190 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854238 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854434 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854408 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854491 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854475 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854546 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854522 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854603 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854582 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854752 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854732 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.854963 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854744 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istiod-ca-cert\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.855115 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854822 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-data\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.855192 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.854962 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-credential-socket\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.855235 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.855189 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-certs\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.856632 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.856610 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-envoy\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.856860 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.856838 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-podinfo\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.880872 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.880843 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-token\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.880968 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.880923 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc6sk\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-kube-api-access-mc6sk\") pod \"e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:55.955001 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:55.954974 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:56.282321 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:56.282293 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv"] Oct 30 20:04:56.283748 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:04:56.283723 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef66d88_ee2e_4c41_aeaa_bf46db26bc89.slice/crio-4a7734b94ccb6c8d6a5a8c9674c261084066b8e9fd8d4ad8319a178ff47d2a24 WatchSource:0}: Error finding container 4a7734b94ccb6c8d6a5a8c9674c261084066b8e9fd8d4ad8319a178ff47d2a24: Status 404 returned error can't find the container with id 4a7734b94ccb6c8d6a5a8c9674c261084066b8e9fd8d4ad8319a178ff47d2a24 Oct 30 20:04:56.286237 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:56.286206 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:56.286305 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:56.286266 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:56.286305 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:56.286298 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:04:56.842858 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:56.842825 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" event={"ID":"aef66d88-ee2e-4c41-aeaa-bf46db26bc89","Type":"ContainerStarted","Data":"d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f"} Oct 30 20:04:56.843273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:56.842878 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" event={"ID":"aef66d88-ee2e-4c41-aeaa-bf46db26bc89","Type":"ContainerStarted","Data":"4a7734b94ccb6c8d6a5a8c9674c261084066b8e9fd8d4ad8319a178ff47d2a24"} Oct 30 20:04:56.914114 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:56.914053 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podStartSLOduration=1.91403261 podStartE2EDuration="1.91403261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:56.912547678 +0000 UTC m=+1013.312719685" watchObservedRunningTime="2026-10-30 20:04:56.91403261 +0000 UTC m=+1013.314204618" Oct 30 20:04:56.955547 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:56.955518 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:04:57.491119 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:57.485494 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:57.711190 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:57.711017 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:57.959650 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:57.959612 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 20:04:57.960154 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:57.959658 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 20:04:58.955968 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:58.955932 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 20:04:58.956167 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:58.956004 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 20:04:59.509925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:59.509365 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:04:59.955345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:59.955307 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 20:04:59.955528 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:04:59.955372 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 20:05:00.956345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:00.956304 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 20:05:00.956757 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:00.956373 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 20:05:01.525946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:01.525915 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:01.956315 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:01.956279 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 20:05:01.956491 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:01.956350 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 20:05:02.956315 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:02.956272 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 20:05:02.956505 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:02.956344 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 20:05:03.374965 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:03.374867 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:03.959576 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:03.959542 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:05:03.960007 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:03.959875 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:05:03.960773 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:03.960750 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:05:05.102072 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:05:05.102032 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:05:05.299270 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:05.299177 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:07.190813 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:07.190775 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:09.521852 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:09.520661 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:09.523225 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:09.523195 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:09.645050 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:09.644997 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:05:10.006373 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.005114 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn"] Oct 30 20:05:10.010956 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.010930 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.028007 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.027791 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn"] Oct 30 20:05:10.106180 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.106137 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.106441 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.106423 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.106608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.106590 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.106710 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.106697 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.106929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.106910 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.107048 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.107034 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed090c9-2774-401e-8965-61385c1e46b0-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.107248 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.107208 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztz86\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-kube-api-access-ztz86\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.107514 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.107498 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.107646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.107630 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed090c9-2774-401e-8965-61385c1e46b0-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.227935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.222779 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.227935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.222838 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed090c9-2774-401e-8965-61385c1e46b0-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.227935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.222868 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztz86\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-kube-api-access-ztz86\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.227935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.222924 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.227935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.222971 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed090c9-2774-401e-8965-61385c1e46b0-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.240631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.239104 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.240631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.239202 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.240631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.239303 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.240631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.239332 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.240631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.239883 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-certs\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.259671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.254995 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-data\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.262627 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.262584 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed090c9-2774-401e-8965-61385c1e46b0-istiod-ca-cert\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.298920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.263629 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-credential-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.582970 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.263678 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:10.582970 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.272487 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-socket\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.582970 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.373358 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:10.616763 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.376776 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv"] Oct 30 20:05:10.645108 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.392703 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-istio-token\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.645293 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.405956 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztz86\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-kube-api-access-ztz86\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.669729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.669647 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-envoy\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.670109 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.670082 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv"] Oct 30 20:05:10.670453 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.670433 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.697602 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.697521 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed090c9-2774-401e-8965-61385c1e46b0-istio-podinfo\") pod \"e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.728734 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.728666 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/708fc137-602b-42fc-9572-7aacdbffa8cb-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.765841 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.765808 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.766125 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.766099 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.766342 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.766323 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.766541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.766524 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.766692 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.766675 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.766853 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.766837 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.767103 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.767084 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54zvr\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-kube-api-access-54zvr\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.767288 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.767273 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.847235 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.843961 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt"] Oct 30 20:05:10.849628 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.849466 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt"] Oct 30 20:05:10.849940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.849594 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.868614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.868582 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.868741 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.868674 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.868741 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.868719 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.868859 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.868770 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.868982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.868867 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54zvr\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-kube-api-access-54zvr\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.868982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.868969 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.869091 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.869061 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/708fc137-602b-42fc-9572-7aacdbffa8cb-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.869154 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.869099 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.869154 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.869128 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.869585 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.869558 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-credential-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.869683 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.869626 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-data\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.869855 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.869831 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-socket\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.870584 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.870560 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-certs\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.871006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.870983 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/708fc137-602b-42fc-9572-7aacdbffa8cb-istiod-ca-cert\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.872610 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.872584 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-envoy\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.873619 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.873593 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-podinfo\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.906922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.906056 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54zvr\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-kube-api-access-54zvr\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.915924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.908939 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-token\") pod \"e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:10.970512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.970479 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.970656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.970532 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.970656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.970624 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.970754 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.970667 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrx6n\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-kube-api-access-rrx6n\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.970754 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.970722 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.970754 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.970738 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.970857 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.970841 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.970927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.970910 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.971068 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.971044 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:10.982087 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.982057 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:10.994181 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:10.994154 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:11.076289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.076253 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077100 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077263 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077317 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077362 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077399 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrx6n\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-kube-api-access-rrx6n\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077437 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077460 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077546 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.077626 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.078078 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.078323 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.078580 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.078975 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.079208 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.083266 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.089919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.088095 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.105075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.105045 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.107577 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.107552 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrx6n\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-kube-api-access-rrx6n\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.173527 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.173497 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:11.383655 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.383623 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn"] Oct 30 20:05:11.388232 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:05:11.388206 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed090c9_2774_401e_8965_61385c1e46b0.slice/crio-a8e465e4d636d1ce5c96a25b03752e9b5ea5bf0130e231a6e86b39015b867b95 WatchSource:0}: Error finding container a8e465e4d636d1ce5c96a25b03752e9b5ea5bf0130e231a6e86b39015b867b95: Status 404 returned error can't find the container with id a8e465e4d636d1ce5c96a25b03752e9b5ea5bf0130e231a6e86b39015b867b95 Oct 30 20:05:11.391209 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.391168 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.391298 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.391258 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.391342 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.391299 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.470201 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:05:11.470171 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708fc137_602b_42fc_9572_7aacdbffa8cb.slice/crio-25e55960b70cbc246edec000a473bfff08ff9d72e9c3b6696d585a43f143193a WatchSource:0}: Error finding container 25e55960b70cbc246edec000a473bfff08ff9d72e9c3b6696d585a43f143193a: Status 404 returned error can't find the container with id 25e55960b70cbc246edec000a473bfff08ff9d72e9c3b6696d585a43f143193a Oct 30 20:05:11.470322 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.470246 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv"] Oct 30 20:05:11.473673 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.473632 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.473802 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.473728 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.473802 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.473776 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.531184 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:05:11.531124 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0964117_5aaf_4b8d_b0b1_ffbfc4496522.slice/crio-659f5900c8f8686d9a04a96418051d43d42cf2845d1a6554eaa02ff65dac95d2 WatchSource:0}: Error finding container 659f5900c8f8686d9a04a96418051d43d42cf2845d1a6554eaa02ff65dac95d2: Status 404 returned error can't find the container with id 659f5900c8f8686d9a04a96418051d43d42cf2845d1a6554eaa02ff65dac95d2 Oct 30 20:05:11.532019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.531756 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt"] Oct 30 20:05:11.534121 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.534080 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.534222 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.534165 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.534222 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.534207 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:11.988940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.987625 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" event={"ID":"9ed090c9-2774-401e-8965-61385c1e46b0","Type":"ContainerStarted","Data":"9f126c64cd60a90eecc036674719338cea7dc22b6ddf5253a04f056863d558db"} Oct 30 20:05:11.988940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.987691 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" event={"ID":"9ed090c9-2774-401e-8965-61385c1e46b0","Type":"ContainerStarted","Data":"a8e465e4d636d1ce5c96a25b03752e9b5ea5bf0130e231a6e86b39015b867b95"} Oct 30 20:05:11.990450 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.990423 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" event={"ID":"d0964117-5aaf-4b8d-b0b1-ffbfc4496522","Type":"ContainerStarted","Data":"76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842"} Oct 30 20:05:11.990520 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.990475 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" event={"ID":"d0964117-5aaf-4b8d-b0b1-ffbfc4496522","Type":"ContainerStarted","Data":"659f5900c8f8686d9a04a96418051d43d42cf2845d1a6554eaa02ff65dac95d2"} Oct 30 20:05:11.993335 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.993306 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" event={"ID":"708fc137-602b-42fc-9572-7aacdbffa8cb","Type":"ContainerStarted","Data":"474c3aba2c753f94581002778892d7ccd031db75ed1ea22eb4cd924ae98d5c12"} Oct 30 20:05:11.993458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:11.993359 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" event={"ID":"708fc137-602b-42fc-9572-7aacdbffa8cb","Type":"ContainerStarted","Data":"25e55960b70cbc246edec000a473bfff08ff9d72e9c3b6696d585a43f143193a"} Oct 30 20:05:12.038513 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.038451 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podStartSLOduration=3.038438412 podStartE2EDuration="3.038438412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:12.036111647 +0000 UTC m=+1028.436283691" watchObservedRunningTime="2026-10-30 20:05:12.038438412 +0000 UTC m=+1028.438610419" Oct 30 20:05:12.078321 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.078277 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:12.079710 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.079682 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 20:05:12.079816 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.079739 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 20:05:12.086688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.086635 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podStartSLOduration=2.086618214 podStartE2EDuration="2.086618214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:12.083630989 +0000 UTC m=+1028.483802997" watchObservedRunningTime="2026-10-30 20:05:12.086618214 +0000 UTC m=+1028.486790222" Oct 30 20:05:12.141411 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.141355 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podStartSLOduration=2.141341071 podStartE2EDuration="2.141341071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:12.137810048 +0000 UTC m=+1028.537982055" watchObservedRunningTime="2026-10-30 20:05:12.141341071 +0000 UTC m=+1028.541513077" Oct 30 20:05:12.174540 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.174502 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:12.198921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.177135 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 20:05:12.198921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.177200 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 20:05:12.671832 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.671622 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:12.987980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:12.987535 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:13.010202 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:13.010166 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 20:05:13.010798 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:13.010764 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 20:05:13.082954 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:13.082490 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 20:05:13.082954 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:13.082552 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 20:05:13.179206 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:13.179175 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 20:05:13.183013 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:13.182969 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 20:05:13.997925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:13.983296 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 20:05:13.997925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:13.983355 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 20:05:14.080920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:14.080263 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 20:05:14.080920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:14.080331 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 20:05:14.174013 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:14.173961 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 20:05:14.174206 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:14.174042 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 20:05:14.988397 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:14.988357 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 20:05:14.988679 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:14.988654 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 20:05:15.079923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:15.079285 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 20:05:15.079923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:15.079346 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 20:05:15.174925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:15.173976 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 20:05:15.174925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:15.174026 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 20:05:15.324479 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:15.324347 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:15.985669 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:15.985623 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 20:05:15.985970 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:15.985941 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 20:05:16.085148 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:16.085113 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 20:05:16.110056 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:16.110019 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 20:05:16.180927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:16.178547 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 20:05:16.180927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:16.178604 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 20:05:17.037925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.031551 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 20:05:17.037925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.031610 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 20:05:17.104405 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.104373 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 20:05:17.104666 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.104637 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 20:05:17.208930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.203078 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 20:05:17.208930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.203147 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 20:05:17.280211 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.279012 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:17.282305 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.280512 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:17.484157 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.484076 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:05:17.607702 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.607645 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:05:17.675434 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.675396 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:05:17.951958 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.941490 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:17.991474 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.990684 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 20:05:17.991474 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:17.990751 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 20:05:18.079331 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:18.079293 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 20:05:18.079656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:18.079602 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 20:05:18.174666 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:18.174627 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 20:05:18.174843 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:18.174700 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 20:05:18.516914 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:18.516862 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:18.983924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:18.983345 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 20:05:18.983924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:18.983406 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 20:05:19.078710 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:19.078679 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 20:05:19.078994 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:19.078966 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 20:05:19.377276 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:19.377191 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:19.378207 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:19.378050 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:19.381661 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:19.381619 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:05:19.988084 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:19.988055 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:19.988578 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:19.988505 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:19.989264 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:19.989241 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:05:20.020986 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:20.020960 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:20.090128 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:20.090099 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:20.090593 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:20.090568 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:20.091718 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:20.091697 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:05:21.860577 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:21.860539 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:23.711420 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:23.711382 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:25.231301 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.231272 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8"] Oct 30 20:05:25.235728 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.235709 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.260867 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.260838 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8"] Oct 30 20:05:25.341478 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341448 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.341638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341487 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.341638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341505 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.341638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341604 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tbz9\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-kube-api-access-4tbz9\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.341765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341753 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.341802 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341782 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.341835 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341825 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.341945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341927 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.342004 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.341974 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443569 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443536 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443710 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443584 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443710 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443655 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443710 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443693 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443859 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443743 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443859 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443826 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443862 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443885 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.443977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.443960 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tbz9\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-kube-api-access-4tbz9\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.444232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.444205 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.444361 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.444338 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.444514 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.444492 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.444617 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.444543 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.444617 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.444593 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.446681 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.446658 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.446879 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.446864 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.469476 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.469453 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.472516 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.472490 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tbz9\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-kube-api-access-4tbz9\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.479763 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.479505 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:25.550891 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.550775 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:25.633115 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.627193 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz"] Oct 30 20:05:25.643162 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.638625 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.665992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.663547 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz"] Oct 30 20:05:25.751156 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751122 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.751343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751169 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.751343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751192 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.751443 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751343 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.751478 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751438 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.751509 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751486 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.751576 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751560 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.751622 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751611 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ks88\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-kube-api-access-2ks88\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.751657 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.751643 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.852982 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853017 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ks88\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-kube-api-access-2ks88\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853045 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853145 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853175 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853193 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853220 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853289 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853556 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853356 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853606 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853573 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-credential-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853715 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853690 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-certs\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853835 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853720 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-socket\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.853835 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.853775 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-data\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.854211 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.854186 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istiod-ca-cert\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.855446 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.855429 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-envoy\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.855792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.855769 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-podinfo\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.889735 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.889676 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ks88\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-kube-api-access-2ks88\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.891535 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.891302 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-token\") pod \"e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:25.895735 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.895713 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8"] Oct 30 20:05:25.896991 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:05:25.896955 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3323b5_cd7c_448d_a483_bd2052af9bdb.slice/crio-a556782616dc30a397936874605a86214fe91b7fadc31f522d579a86db82aa87 WatchSource:0}: Error finding container a556782616dc30a397936874605a86214fe91b7fadc31f522d579a86db82aa87: Status 404 returned error can't find the container with id a556782616dc30a397936874605a86214fe91b7fadc31f522d579a86db82aa87 Oct 30 20:05:25.900304 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.900257 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:25.900401 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.900346 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:25.900401 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.900389 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:25.971769 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:25.971743 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:26.311699 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.311662 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz"] Oct 30 20:05:26.323602 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:05:26.323575 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a8ade9_4b09_47c2_b7f5_ec0d6ad8f5be.slice/crio-181aa933ee9a75a833202d83a63baac372663c7a9e050eda05bf7d640ba78826 WatchSource:0}: Error finding container 181aa933ee9a75a833202d83a63baac372663c7a9e050eda05bf7d640ba78826: Status 404 returned error can't find the container with id 181aa933ee9a75a833202d83a63baac372663c7a9e050eda05bf7d640ba78826 Oct 30 20:05:26.326217 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.326184 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" event={"ID":"0e3323b5-cd7c-448d-a483-bd2052af9bdb","Type":"ContainerStarted","Data":"a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4"} Oct 30 20:05:26.326318 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.326253 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" event={"ID":"0e3323b5-cd7c-448d-a483-bd2052af9bdb","Type":"ContainerStarted","Data":"a556782616dc30a397936874605a86214fe91b7fadc31f522d579a86db82aa87"} Oct 30 20:05:26.327153 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.327112 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:26.327241 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.327204 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:26.327283 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.327244 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:05:26.377127 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.377083 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podStartSLOduration=1.377067094 podStartE2EDuration="1.377067094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:26.374474558 +0000 UTC m=+1042.774646566" watchObservedRunningTime="2026-10-30 20:05:26.377067094 +0000 UTC m=+1042.777239100" Oct 30 20:05:26.551388 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.551349 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:26.553411 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.553386 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 20:05:26.553536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:26.553437 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 20:05:27.017923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.017611 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:27.364631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.364546 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" event={"ID":"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be","Type":"ContainerStarted","Data":"f0c413698473d8574218c1a2799e6f7ac85f978b2b669674d4814836a17e13ab"} Oct 30 20:05:27.364631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.364601 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" event={"ID":"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be","Type":"ContainerStarted","Data":"181aa933ee9a75a833202d83a63baac372663c7a9e050eda05bf7d640ba78826"} Oct 30 20:05:27.431919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.431083 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podStartSLOduration=2.431065119 podStartE2EDuration="2.431065119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:27.418120932 +0000 UTC m=+1043.818292934" watchObservedRunningTime="2026-10-30 20:05:27.431065119 +0000 UTC m=+1043.831237127" Oct 30 20:05:27.475958 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.475851 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:27.568482 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.568452 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 20:05:27.568762 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.568731 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 20:05:27.972956 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.972650 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:27.974190 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.974166 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:27.974272 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:27.974214 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:28.553300 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:28.553256 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 20:05:28.553793 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:28.553332 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 20:05:28.973042 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:28.973008 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:28.973220 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:28.973062 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:29.552149 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:29.552117 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 20:05:29.552397 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:29.552370 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 20:05:29.641746 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:29.641709 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:29.972961 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:29.972928 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:29.973136 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:29.972991 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:30.551387 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:30.551352 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 20:05:30.551568 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:30.551423 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 20:05:30.972907 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:30.972855 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:30.973310 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:30.972922 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:31.552349 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:31.552309 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 20:05:31.552550 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:31.552375 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 20:05:31.736732 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:31.736697 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:31.972412 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:31.972378 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:31.972652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:31.972632 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:32.552576 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:32.552527 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 20:05:32.553181 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:32.553134 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 20:05:32.983168 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:32.983129 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:32.983442 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:32.983414 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:33.249843 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:33.249738 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:33.493241 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:33.493196 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:33.556983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:33.555300 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 20:05:33.556983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:33.555373 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 20:05:33.654095 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:33.654046 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:33.974341 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:33.974303 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:33.974604 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:33.974578 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:34.046223 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.045022 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:34.254980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.250337 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:34.254980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.251115 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:05:34.266445 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.265991 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:34.691982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.628255 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:05:34.825164 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.742277 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:34.825164 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.742875 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:05:34.825164 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.745449 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:34.825164 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.785444 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:05:34.876530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:34.876494 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:05:35.228072 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.202102 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:35.361145 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.305555 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:35.362836 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.309264 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": context deadline exceeded" Oct 30 20:05:35.378624 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.309301 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:05:35.380252 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.309654 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:35.380518 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.380472 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:35.381758 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.370207 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:05:35.382526 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.346302 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:35.382722 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.382684 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:35.382885 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.346444 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:05:35.384436 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.384418 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:05:35.835019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.831418 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:05:35.864407 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:05:35.852538 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:05:35.920926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.894564 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:05:35.961205 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.932997 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:35.962789 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.962176 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:35.962789 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.962238 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:36.018934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:35.971542 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:05:36.018934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.003978 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 20:05:36.018934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.004028 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 20:05:36.113978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.021681 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:05:36.113978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.021741 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:05:36.204172 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.204133 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:36.230951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.225971 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:36.384984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.364976 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:36.393927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.385244 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:36.393927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.385660 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:05:36.685917 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.685800 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:36.796190 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.796156 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:36.835525 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.835487 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:05:36.900050 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.899992 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:05:36.936166 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.936031 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:36.977439 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:36.977395 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:37.036995 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.036957 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:37.052924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.037216 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:37.387160 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.387112 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:05:37.388218 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.388174 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:37.421505 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.421467 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:37.806351 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.805063 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:37.806351 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.805132 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:37.837352 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.836980 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:37.901602 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.901562 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:05:37.959332 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.959292 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:05:37.993253 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:37.992233 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:38.032514 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.032160 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:38.032514 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.032220 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:38.154873 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.154031 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:38.154873 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.154102 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:38.387920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.387734 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:38.403679 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.391490 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:38.430989 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.430215 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:38.501928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.500992 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:05:38.581581 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.581552 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:05:38.606840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.606811 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:05:38.620255 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.620231 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:05:38.620485 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.620473 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:05:38.676267 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.676238 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:05:38.693763 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.693273 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:05:38.761080 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.761045 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:38.776049 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.762985 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:38.776282 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.776178 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:05:38.873979 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.873950 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:38.874959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.874937 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:38.875615 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:38.875591 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:05:39.244156 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:39.244106 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:40.863551 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:40.863519 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx"] Oct 30 20:05:40.869817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:40.869795 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:40.888580 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:40.888554 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx"] Oct 30 20:05:40.896337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:40.896306 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-efbac3\"/\"distinct-w2-efbac3-kserve-self-signed-certs\"" Oct 30 20:05:40.902677 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:40.902655 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:41.053330 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.053300 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcwb2\" (UniqueName: \"kubernetes.io/projected/0b7d73fb-1fb8-4969-a783-f242346001f2-kube-api-access-hcwb2\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.053515 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.053369 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-tmp-dir\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.053515 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.053494 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-model-cache\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.053659 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.053642 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-dshm\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.053711 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.053699 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7d73fb-1fb8-4969-a783-f242346001f2-tls-certs\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.053853 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.053835 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-home\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.155352 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.155319 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-tmp-dir\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.155849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.155388 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-model-cache\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.155849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.155571 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-dshm\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.155849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.155632 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7d73fb-1fb8-4969-a783-f242346001f2-tls-certs\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.155849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.155779 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-home\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.156100 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.155847 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-model-cache\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.156100 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.155868 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcwb2\" (UniqueName: \"kubernetes.io/projected/0b7d73fb-1fb8-4969-a783-f242346001f2-kube-api-access-hcwb2\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.156166 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.156134 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-home\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.156482 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.156461 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-tmp-dir\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.160335 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.160290 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7d73fb-1fb8-4969-a783-f242346001f2-tls-certs\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.162506 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.162477 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-dshm\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.182368 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.182338 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcwb2\" (UniqueName: \"kubernetes.io/projected/0b7d73fb-1fb8-4969-a783-f242346001f2-kube-api-access-hcwb2\") pod \"distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.184108 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.184082 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:41.275285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.275257 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:41.501226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.501129 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx"] Oct 30 20:05:41.517141 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:05:41.517067 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b7d73fb_1fb8_4969_a783_f242346001f2.slice/crio-bf16c37ab61c235ec4d71cde81b063e9de0788afd903448d0ec9c4856e6ff239 WatchSource:0}: Error finding container bf16c37ab61c235ec4d71cde81b063e9de0788afd903448d0ec9c4856e6ff239: Status 404 returned error can't find the container with id bf16c37ab61c235ec4d71cde81b063e9de0788afd903448d0ec9c4856e6ff239 Oct 30 20:05:41.802882 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.802802 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" event={"ID":"0b7d73fb-1fb8-4969-a783-f242346001f2","Type":"ContainerStarted","Data":"5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765"} Oct 30 20:05:41.802882 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.802847 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" event={"ID":"0b7d73fb-1fb8-4969-a783-f242346001f2","Type":"ContainerStarted","Data":"bf16c37ab61c235ec4d71cde81b063e9de0788afd903448d0ec9c4856e6ff239"} Oct 30 20:05:41.850660 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:41.850601 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podStartSLOduration=1.850582187 podStartE2EDuration="1.850582187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:41.849113331 +0000 UTC m=+1058.249285340" watchObservedRunningTime="2026-10-30 20:05:41.850582187 +0000 UTC m=+1058.250754197" Oct 30 20:05:42.185007 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:42.184977 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:43.122523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:43.122489 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:44.914130 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:44.914098 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:46.759964 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:46.759934 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:48.602406 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:48.602376 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:50.841924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:50.839482 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:51.194925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:51.194400 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:05:52.002923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:51.993002 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:52.203599 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:52.203546 6411 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:52.421044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:52.420988 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:52.421688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:52.421654 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:52.442928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:52.438649 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:05:52.668761 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:52.666998 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:52.916925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:52.902003 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:53.005929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:52.994053 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:53.040930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.035025 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:05:53.192494 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.006064 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:53.507984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.467187 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:53.507984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.467819 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:05:53.511920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.509735 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:05:53.511920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.510041 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:05:53.511920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.510251 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:05:53.511920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.510304 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:05:53.677927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.669997 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:53.678268 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.678243 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:05:53.915922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.915394 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:53.942928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:53.934753 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:05:54.181977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.161958 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:05:54.274928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.272755 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:05:54.274928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.273200 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:05:54.430924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.426437 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:54.430924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.426517 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:54.476983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.473393 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:05:54.516136 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.516074 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:54.525953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.516735 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:54.714936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.709106 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:54.850327 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.847438 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:54.850327 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.847499 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:54.929155 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.929113 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:05:54.965443 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.965394 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:05:54.965751 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:54.965703 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:55.192472 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:55.192429 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:55.240568 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:55.193065 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:55.259946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:55.239181 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:55.463980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:55.454859 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:55.596970 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:55.595993 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:55.596970 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:55.596507 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:05:55.596970 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:55.596761 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:05:55.836974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:55.720760 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 20:05:56.047768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:56.047738 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:05:56.105405 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:56.105323 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:56.108018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:56.107993 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:05:56.109162 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:56.109143 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:05:56.109553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:56.109524 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:05:56.109996 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:56.109972 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:05:56.115840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:56.114060 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:05:56.115840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:56.115270 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:05:57.607700 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:57.607641 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:05:59.496998 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:05:59.496961 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:00.726019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.725984 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw"] Oct 30 20:06:00.730274 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.730247 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.747940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.747913 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 30 20:06:00.748075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.747950 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 30 20:06:00.748075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.748019 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r79wj\"" Oct 30 20:06:00.759547 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.759524 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 30 20:06:00.763516 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.763493 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw"] Oct 30 20:06:00.808408 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.808367 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ll6t\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-kube-api-access-8ll6t\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.808602 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.808503 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.808602 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.808550 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.808602 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.808591 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.808742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.808617 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.808742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.808704 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.808913 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.808871 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee40c7b0-1679-4983-a004-ddc225786c80-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.910759 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.910721 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ll6t\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-kube-api-access-8ll6t\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.911267 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.911239 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.911520 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.911502 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.911969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.911949 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.912498 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.912479 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.913404 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.913376 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.919477 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.919450 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee40c7b0-1679-4983-a004-ddc225786c80-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.920083 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.920026 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.920083 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.913306 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.921567 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.921542 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.922031 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.922002 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee40c7b0-1679-4983-a004-ddc225786c80-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.922520 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.922502 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.936246 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.936221 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ll6t\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-kube-api-access-8ll6t\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:00.937012 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:00.936988 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nwmhw\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:01.044749 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:01.044489 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:01.196753 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:01.196723 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:06:01.210006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:01.209974 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:06:01.377943 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:01.377884 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw"] Oct 30 20:06:01.379047 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:06:01.379018 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee40c7b0_1679_4983_a004_ddc225786c80.slice/crio-e8f724faa4f6a5be0b0c718eeaaf762959dfde6cd4936af51fde1b420a4d5fda WatchSource:0}: Error finding container e8f724faa4f6a5be0b0c718eeaaf762959dfde6cd4936af51fde1b420a4d5fda: Status 404 returned error can't find the container with id e8f724faa4f6a5be0b0c718eeaaf762959dfde6cd4936af51fde1b420a4d5fda Oct 30 20:06:01.381346 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:01.381324 6411 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 20:06:01.605418 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:01.605383 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:02.151140 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:02.151105 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" event={"ID":"ee40c7b0-1679-4983-a004-ddc225786c80","Type":"ContainerStarted","Data":"e8f724faa4f6a5be0b0c718eeaaf762959dfde6cd4936af51fde1b420a4d5fda"} Oct 30 20:06:03.623434 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:03.623399 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:04.357705 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:04.357672 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:06:04.357804 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:04.357738 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:06:05.112103 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:06:05.112063 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:06:05.217919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:05.216989 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" event={"ID":"ee40c7b0-1679-4983-a004-ddc225786c80","Type":"ContainerStarted","Data":"167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6"} Oct 30 20:06:05.221952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:05.219158 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:05.221952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:05.220316 6411 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-nwmhw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 20:06:05.221952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:05.220370 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" podUID="ee40c7b0-1679-4983-a004-ddc225786c80" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 20:06:05.285652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:05.285587 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" podStartSLOduration=2.310768156 podStartE2EDuration="5.285569574s" totalImagesPullingTime="2.974801418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:06:05.28400003 +0000 UTC m=+1081.684172035" watchObservedRunningTime="2026-10-30 20:06:05.285569574 +0000 UTC m=+1081.685741583" Oct 30 20:06:05.607634 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:05.607598 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:06.226545 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:06.226517 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:06:07.416908 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:07.416866 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:09.253641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:09.253605 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:10.440330 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.440299 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr"] Oct 30 20:06:10.444293 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.444262 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.491521 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.491484 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr"] Oct 30 20:06:10.512487 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.512452 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.512692 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.512675 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.513361 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.513329 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.513461 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.513433 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbrqt\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-kube-api-access-jbrqt\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.513502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.513471 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.513540 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.513503 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.513580 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.513544 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.513616 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.513606 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.513696 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.513677 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.615651 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.615617 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.615811 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.615665 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.615811 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.615783 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616020 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.615865 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbrqt\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-kube-api-access-jbrqt\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616020 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.615941 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616333 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.616310 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-data\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616447 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.615975 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616515 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.616495 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616515 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.616495 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-credential-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616626 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.616520 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istiod-ca-cert\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616626 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.616598 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616746 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.616676 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.616798 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.616758 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-certs\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.617217 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.617189 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-socket\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.618532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.618507 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-envoy\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.619300 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.619282 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-podinfo\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.641082 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.641057 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-token\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.641630 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.641606 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbrqt\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-kube-api-access-jbrqt\") pod \"e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:10.761790 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:10.761718 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:11.105143 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.102095 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr"] Oct 30 20:06:11.105545 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:06:11.105511 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76a7e4b_02f3_448e_b296_8ee0ef7d90a1.slice/crio-4abfa7d64f4c79194d5995f84c1b1f91596a514e5071ad73d9d0b7a55cc0f9e4 WatchSource:0}: Error finding container 4abfa7d64f4c79194d5995f84c1b1f91596a514e5071ad73d9d0b7a55cc0f9e4: Status 404 returned error can't find the container with id 4abfa7d64f4c79194d5995f84c1b1f91596a514e5071ad73d9d0b7a55cc0f9e4 Oct 30 20:06:11.108761 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.108711 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:06:11.108959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.108880 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:06:11.109065 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.109043 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:06:11.166637 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.166597 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:11.270213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.270176 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" event={"ID":"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1","Type":"ContainerStarted","Data":"74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1"} Oct 30 20:06:11.270410 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.270233 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" event={"ID":"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1","Type":"ContainerStarted","Data":"4abfa7d64f4c79194d5995f84c1b1f91596a514e5071ad73d9d0b7a55cc0f9e4"} Oct 30 20:06:11.338225 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.338145 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podStartSLOduration=1.338127172 podStartE2EDuration="1.338127172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:06:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:06:11.335249729 +0000 UTC m=+1087.735421737" watchObservedRunningTime="2026-10-30 20:06:11.338127172 +0000 UTC m=+1087.738299181" Oct 30 20:06:11.388310 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.388245 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:11.762160 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:11.762086 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:12.763019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:12.762985 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:12.763464 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:12.763039 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:13.762915 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:13.762861 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:13.763077 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:13.762949 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:13.763844 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:13.763821 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:14.762761 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:14.762729 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:14.762965 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:14.762774 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:15.794923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:15.789419 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:15.794923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:15.789738 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:16.155986 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.120360 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:06:16.267956 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.197813 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:16.403857 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.234295 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:16.537281 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.537179 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:16.663976 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.663925 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:16.698251 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.664280 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:16.768040 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.767995 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:06:16.817039 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.770741 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:16.817592 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.817559 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:16.817745 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:16.792830 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": context deadline exceeded" Oct 30 20:06:17.122991 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.122267 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:17.122991 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.122868 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:06:17.126920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.123467 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:06:17.430121 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.430084 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:17.540119 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.540070 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:17.541121 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.541095 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:06:17.586541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.586502 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:06:17.586932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.586879 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:06:17.587527 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.587467 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:06:17.587748 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.587731 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:06:17.765149 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.765056 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:17.880922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.874995 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:06:17.880922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.788257 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:17.880922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.875589 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:17.880922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.802175 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:17.880922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.875880 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:06:17.880922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.788517 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:06:17.880922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:17.877322 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:06:18.147244 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.147204 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:06:18.174030 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.147495 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:18.175345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.175322 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:06:18.511567 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.511469 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:18.511925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.511873 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:18.562924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.547449 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:18.594925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.587005 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:18.594925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.587343 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:18.764393 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.763877 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:18.764393 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.763957 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:18.902924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.893315 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:06:18.902924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.893609 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:18.902924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:18.893800 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:06:19.178939 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:19.178425 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:06:19.194923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:19.180107 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:06:19.564592 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:19.563350 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:19.589987 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:19.589268 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:19.589987 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:19.589667 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:06:19.991605 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:19.924016 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.120250 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.072367 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:20.120250 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.072443 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:20.158950 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.151067 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:20.158950 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.154768 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:06:20.188946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.162984 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:06:20.188946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.179987 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.188946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.180462 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.219940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.216048 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:20.219940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.216127 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.234435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.220301 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:20.234435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.220383 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.234435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.221022 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:20.234435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.221062 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.253885 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.251400 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:20.253885 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.251484 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.338104 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.338066 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:20.338447 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.338417 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.577986 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.563679 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.730542 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.730020 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:20.801922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.769682 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:06:20.801922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.770068 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:06:20.922924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.922342 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:20.922924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:20.922405 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:21.038457 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.038420 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:21.039057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.039023 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:21.179094 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.174960 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:21.179094 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.175385 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:21.179094 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.175676 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:21.180707 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.180672 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:21.181188 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.181162 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:06:21.292318 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.292272 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:21.293544 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.292566 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:21.351011 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.350983 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:06:21.596045 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.591243 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:06:21.773929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.768605 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:21.773929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.768669 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:21.798928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.785475 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:06:21.798928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:21.785761 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:06:22.178930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.176248 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:22.178930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.177644 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:22.191934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.184040 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:22.192236 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.192186 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:06:22.216934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.205975 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:22.311922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.287561 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:06:22.311922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.289294 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:06:22.488286 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.487007 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:06:22.531323 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.531297 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:06:22.606247 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.606195 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:22.610974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.609432 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:22.806510 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.806414 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Oct 30 20:06:22.806806 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.806774 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Oct 30 20:06:22.836400 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.833948 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:06:22.836400 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.834020 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:06:22.840785 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:22.840672 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:06:23.962050 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:23.962021 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:23.962591 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:23.962403 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:23.963291 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:23.963273 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:06:24.752538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:24.752503 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:26.568305 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:26.568258 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:27.688781 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:27.688744 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:28.533010 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:28.520719 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:30.498041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:30.498007 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:32.374956 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:32.374924 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:34.304046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:34.303307 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:35.099266 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:06:35.099229 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:06:36.235397 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:36.235367 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:38.214937 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:38.214874 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:40.031453 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:40.031328 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:41.808927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:41.790764 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:42.241926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:42.241569 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:42.242148 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:42.242064 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/health\": context deadline exceeded" Oct 30 20:06:42.498140 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:42.498026 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:06:42.593047 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:42.592998 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:42.655499 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:42.655457 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:42.903491 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:42.901631 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:42.925067 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:42.925010 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:06:42.927991 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:42.925210 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:43.246985 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.244008 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:06:43.267777 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.267729 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:43.268509 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.268487 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:06:43.581032 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.550271 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:43.626956 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.603101 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:06:43.651924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.630817 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:43.651924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.631828 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:06:43.671382 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.671331 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:06:43.690060 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.674733 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:06:43.935982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.933218 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:43.943805 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.933945 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:43.947464 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:43.933995 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:44.380995 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.316677 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:06:44.380995 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.360049 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:06:44.550923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.539497 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:06:44.581602 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.581571 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:06:44.581861 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.581849 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:06:44.602668 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.602636 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:06:44.623923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.612545 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:44.662732 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.662630 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:06:44.815536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:44.695125 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:45.002352 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.002241 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:45.019037 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.018984 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:06:45.026928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.026782 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:06:45.183236 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.183190 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:45.408983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.382588 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": context deadline exceeded" Oct 30 20:06:45.553682 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.496375 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:45.581042 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.564400 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:45.581765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.573626 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:06:45.621943 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.620982 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:06:45.816138 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.816031 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:45.859000 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:45.846110 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:46.005400 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.005357 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:46.105979 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.103105 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:46.142208 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.142165 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:06:46.585923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.585226 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:46.617166 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.613849 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:46.663867 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.662175 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:06:46.713935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.690207 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": context deadline exceeded" Oct 30 20:06:46.858981 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.828062 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:06:46.888762 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:46.887747 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:47.051936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:47.009991 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:47.113194 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:47.113066 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:06:47.165289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:47.165249 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:06:47.676976 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:47.673173 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:47.765925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:47.753532 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:06:47.765925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:47.761497 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:47.833343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:47.832401 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:48.142031 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.122074 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:48.167514 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.166175 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:48.193088 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.171686 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:06:48.231978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.231932 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:06:48.694175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.694131 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:48.766366 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.766320 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:48.823923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.775665 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:06:48.841453 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.841416 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:48.970048 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.963073 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:48.970048 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.858592 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:06:49.024973 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.852025 6411 patch_prober.go:29] interesting pod/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:06:49.024973 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:48.983467 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:49.053032 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.052954 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:06:49.150353 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.150307 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:49.187736 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.187703 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:06:49.188767 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.188742 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:06:49.192283 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.192240 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:06:49.196782 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.196758 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:06:49.211600 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.211556 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:49.242982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.227213 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:06:49.242982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.228213 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:06:49.298491 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.298462 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:06:49.344646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.344612 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:06:49.349235 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:49.349170 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:06:51.323092 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:51.323061 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:53.242737 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:53.242698 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:55.192483 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:55.192442 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:57.340454 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:57.340387 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:06:59.334482 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:06:59.334452 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:01.469520 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:01.469483 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:03.355064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:03.354993 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:03.688044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:03.687991 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:03.800840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:03.800798 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:07:03.825321 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:03.825279 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:03.826070 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:03.826040 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:04.037393 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.037268 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:07:04.063926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.038748 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:07:04.138977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.138934 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:07:04.363166 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.363047 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:04.364462 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.364440 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:07:04.808925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.808707 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:07:04.808925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.808783 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:04.863098 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.863036 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:04.894734 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.884987 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:04.894734 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.885269 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:04.894734 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.885481 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:04.894734 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.885535 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:07:04.894734 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.885571 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:07:04.918926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.908939 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:04.918926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:04.912107 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:07:05.068006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.062548 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:07:05.068006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.063460 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:07:05.075494 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.075456 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:05.090949 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.076479 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:07:05.141945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.135990 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:07:05.141945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.136054 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:05.146085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.146044 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:05.151302 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.146382 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:07:05.218924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.214990 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:05.252004 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.221716 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:05.252981 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.241406 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:07:05.253195 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.253165 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:05.403928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.396658 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded" Oct 30 20:07:05.800981 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.778653 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:05.946531 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.946478 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:05.964924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.948840 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:07:05.996925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.966964 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:05.996925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:05.967936 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:07:06.119445 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.119351 6411 patch_prober.go:29] interesting pod/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:07:06.119702 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.119671 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:06.120266 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.120236 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:06.127196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.127164 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:06.151297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.151261 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:06.164923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.154754 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:06.165390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.165358 6411 patch_prober.go:29] interesting pod/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:07:06.165576 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.165553 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:06.251955 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:07:06.237946 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:07:06.427374 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.419197 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:06.671967 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.671926 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:06.952063 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.952020 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded" Oct 30 20:07:06.965001 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.964952 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:07:06.976078 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.976033 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:07:06.984045 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:06.984007 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:07:07.125074 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.125028 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:07.181982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.181941 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:07:07.189932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.186279 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:07:07.465323 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.465268 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:07.896048 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.862005 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:07:07.896048 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.862072 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:07.969090 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.969038 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:07:07.970395 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.970362 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:07.996043 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.995994 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:08.026918 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:07.997042 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:07:08.137925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:08.117197 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:08.137925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:08.127345 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:07:08.188060 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:08.187455 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:08.233038 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:08.232987 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:07:08.499833 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:08.499602 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:08.995924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:08.986011 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:08.995924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:08.987355 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:09.055076 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.055024 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:09.056463 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.056431 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:07:09.149922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.142324 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:07:09.242915 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.242838 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:09.246675 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.244461 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:09.253706 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.253679 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:07:09.312030 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.310993 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:07:09.346425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.346397 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:09.353927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.353884 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:07:09.388044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.385953 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:07:09.411247 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.410412 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:07:09.432411 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.432383 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:07:09.433300 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:09.433279 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:07:10.470166 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:10.470125 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:10.729478 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:10.729397 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht"] Oct 30 20:07:10.868942 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:10.868880 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:11.469066 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:11.468999 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:11.729403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:11.729316 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:12.134027 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:12.133971 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:12.777034 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:12.776993 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:14.911405 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:14.911365 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:16.542572 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:16.542534 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb"] Oct 30 20:07:16.655956 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:16.655920 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:16.921492 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:16.921455 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:17.291344 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:17.291262 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:17.724544 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:17.722326 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:18.079443 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:18.079362 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:19.149626 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:19.149588 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:21.042265 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:21.042221 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:22.040326 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:22.040288 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:22.281088 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:22.281048 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c"] Oct 30 20:07:22.457080 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:22.454955 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:22.686435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:22.686353 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:23.408982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:23.408945 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:23.597567 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:23.597532 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:25.575841 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:25.575807 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:25.757638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:25.757595 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" containerID="cri-o://bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185" gracePeriod=45 Oct 30 20:07:26.050302 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.050278 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:07:26.096518 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.096482 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-home\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-home\") pod \"93138c42-74ff-4d13-b362-4f859beb67df\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " Oct 30 20:07:26.096723 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.096702 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93138c42-74ff-4d13-b362-4f859beb67df-kube-api-access-v72xb\" (UniqueName: \"kubernetes.io/projected/93138c42-74ff-4d13-b362-4f859beb67df-kube-api-access-v72xb\") pod \"93138c42-74ff-4d13-b362-4f859beb67df\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " Oct 30 20:07:26.096803 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.096743 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93138c42-74ff-4d13-b362-4f859beb67df-tls-certs\" (UniqueName: \"kubernetes.io/secret/93138c42-74ff-4d13-b362-4f859beb67df-tls-certs\") pod \"93138c42-74ff-4d13-b362-4f859beb67df\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " Oct 30 20:07:26.096803 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.096793 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-model-cache\") pod \"93138c42-74ff-4d13-b362-4f859beb67df\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " Oct 30 20:07:26.096948 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.096863 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-tmp-dir\") pod \"93138c42-74ff-4d13-b362-4f859beb67df\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " Oct 30 20:07:26.097018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.096952 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-dshm\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-dshm\") pod \"93138c42-74ff-4d13-b362-4f859beb67df\" (UID: \"93138c42-74ff-4d13-b362-4f859beb67df\") " Oct 30 20:07:26.099886 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.099847 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-model-cache" pod "93138c42-74ff-4d13-b362-4f859beb67df" (UID: "93138c42-74ff-4d13-b362-4f859beb67df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:26.100080 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.099942 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-tmp-dir" pod "93138c42-74ff-4d13-b362-4f859beb67df" (UID: "93138c42-74ff-4d13-b362-4f859beb67df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:26.100174 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.100121 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-home" pod "93138c42-74ff-4d13-b362-4f859beb67df" (UID: "93138c42-74ff-4d13-b362-4f859beb67df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:26.123664 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.123635 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93138c42-74ff-4d13-b362-4f859beb67df-tls-certs" pod "93138c42-74ff-4d13-b362-4f859beb67df" (UID: "93138c42-74ff-4d13-b362-4f859beb67df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:26.123969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.123939 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-dshm" pod "93138c42-74ff-4d13-b362-4f859beb67df" (UID: "93138c42-74ff-4d13-b362-4f859beb67df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:26.127318 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.127247 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93138c42-74ff-4d13-b362-4f859beb67df-kube-api-access-v72xb" pod "93138c42-74ff-4d13-b362-4f859beb67df" (UID: "93138c42-74ff-4d13-b362-4f859beb67df"). InnerVolumeSpecName "kube-api-access-v72xb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:26.199959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.199879 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:26.199959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.199964 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v72xb\" (UniqueName: \"kubernetes.io/projected/93138c42-74ff-4d13-b362-4f859beb67df-kube-api-access-v72xb\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:26.200343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.199981 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93138c42-74ff-4d13-b362-4f859beb67df-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:26.200343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.199998 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:26.200343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.200032 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:26.200343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.200044 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93138c42-74ff-4d13-b362-4f859beb67df-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:26.546016 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.545975 6411 generic.go:356] "Generic (PLEG): container finished" podID="93138c42-74ff-4d13-b362-4f859beb67df" containerID="bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185" exitCode=0 Oct 30 20:07:26.546231 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.546108 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" event={"ID":"93138c42-74ff-4d13-b362-4f859beb67df","Type":"ContainerDied","Data":"bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185"} Oct 30 20:07:26.546231 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.546146 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" event={"ID":"93138c42-74ff-4d13-b362-4f859beb67df","Type":"ContainerDied","Data":"b178e37eb6301122f35891e6193d9f177e11e6aac6b3d63c56bc5d7f9b1c1382"} Oct 30 20:07:26.546231 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.546168 6411 scope.go:122] "RemoveContainer" containerID="bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185" Oct 30 20:07:26.546397 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.546330 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" Oct 30 20:07:26.564378 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.564356 6411 scope.go:122] "RemoveContainer" containerID="bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185" Oct 30 20:07:26.564744 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:07:26.564723 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185\": container with ID starting with bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185 not found: ID does not exist" containerID="bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185" Oct 30 20:07:26.564800 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.564754 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185"} err="failed to get container status \"bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185\": rpc error: code = NotFound desc = could not find container \"bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185\": container with ID starting with bc2b2ac345d28470bf8d6f70f324ead28cda37db81c9d906052d3dd70b97f185 not found: ID does not exist" Oct 30 20:07:26.613575 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.613541 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht"] Oct 30 20:07:26.619641 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.619614 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht"] Oct 30 20:07:26.947425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:26.947349 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-912f24/distinct-w3-912f24-kserve-7f44b47cc8-4sjht" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:27.288119 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:27.287861 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" podUID="5c095837-e094-43c1-aa14-5715094b2bbb" containerName="payload-processing" containerID="cri-o://57a6fe4b1a817535b7bf93e49682a4253f5405cb800c8a97c37695c1df515211" gracePeriod=25 Oct 30 20:07:27.650109 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:27.650078 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:27.736209 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:27.736179 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93138c42-74ff-4d13-b362-4f859beb67df" path="/var/lib/kubelet/pods/93138c42-74ff-4d13-b362-4f859beb67df/volumes" Oct 30 20:07:30.283922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:30.279657 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:30.681060 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:30.681008 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:31.055986 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:31.046002 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:31.074932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:31.061203 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:31.139181 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:31.112155 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:07:31.295071 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:31.285020 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": context deadline exceeded" Oct 30 20:07:31.295071 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:31.285153 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:07:31.295071 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:31.286320 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:07:31.396749 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:31.372503 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:31.586923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:31.586078 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" containerID="cri-o://9d53bd631d2dc7d744fa5fa9e7a035691ac46c4e95b8d708508f98f8d1a5928e" gracePeriod=45 Oct 30 20:07:32.071925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.070994 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:32.098675 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.098625 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:07:32.149054 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.116035 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:32.149456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.116683 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:07:32.150444 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.150426 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:07:32.150647 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.150615 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:32.193149 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.193088 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:32.312530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.312486 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:32.504777 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.409373 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded" Oct 30 20:07:32.591940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.409406 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:07:32.626948 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.592353 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:07:32.626948 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.430867 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:32.626948 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.594148 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:07:32.626948 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.594429 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": dial tcp 10.133.0.67:8000: connect: connection refused" Oct 30 20:07:32.689839 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.672695 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:07:32.759354 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.759250 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" podUID="5c095837-e094-43c1-aa14-5715094b2bbb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 20:07:32.830243 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:32.826273 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:33.158152 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:33.080335 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:33.158152 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:33.129989 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:33.158152 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:33.154776 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:07:33.343676 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:33.342702 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": dial tcp 10.133.0.67:8000: connect: connection refused" Oct 30 20:07:33.565976 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:33.564039 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:07:33.565976 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:33.564835 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:07:33.677109 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:33.676016 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:33.830599 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:33.830494 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:34.096225 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.096118 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:07:34.137268 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.137237 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:07:34.171744 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.167403 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded" Oct 30 20:07:34.175675 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.169398 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:34.250923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.246641 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:07:34.269705 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.269674 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:07:34.281546 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.281509 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:07:34.290237 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.290209 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:07:34.319321 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.319287 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": dial tcp 10.133.0.67:8000: connect: connection refused" Oct 30 20:07:34.322595 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.322573 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:34.374612 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.373400 6411 generic.go:356] "Generic (PLEG): container finished" podID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerID="9d53bd631d2dc7d744fa5fa9e7a035691ac46c4e95b8d708508f98f8d1a5928e" exitCode=0 Oct 30 20:07:34.374612 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:34.373449 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" event={"ID":"4f940223-245c-4132-880c-dd2f75dfe9cf","Type":"ContainerDied","Data":"9d53bd631d2dc7d744fa5fa9e7a035691ac46c4e95b8d708508f98f8d1a5928e"} Oct 30 20:07:35.028507 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.028473 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:35.098742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.098715 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:07:35.099223 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:07:35.099190 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:07:35.193343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.193244 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f940223-245c-4132-880c-dd2f75dfe9cf-kube-api-access-74nvh\" (UniqueName: \"kubernetes.io/projected/4f940223-245c-4132-880c-dd2f75dfe9cf-kube-api-access-74nvh\") pod \"4f940223-245c-4132-880c-dd2f75dfe9cf\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " Oct 30 20:07:35.193343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.193297 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-dshm\") pod \"4f940223-245c-4132-880c-dd2f75dfe9cf\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " Oct 30 20:07:35.193343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.193329 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f940223-245c-4132-880c-dd2f75dfe9cf-tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940223-245c-4132-880c-dd2f75dfe9cf-tls-certs\") pod \"4f940223-245c-4132-880c-dd2f75dfe9cf\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " Oct 30 20:07:35.193625 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.193467 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-tmp-dir\") pod \"4f940223-245c-4132-880c-dd2f75dfe9cf\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " Oct 30 20:07:35.193625 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.193604 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-home\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-home\") pod \"4f940223-245c-4132-880c-dd2f75dfe9cf\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " Oct 30 20:07:35.193730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.193677 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-model-cache\") pod \"4f940223-245c-4132-880c-dd2f75dfe9cf\" (UID: \"4f940223-245c-4132-880c-dd2f75dfe9cf\") " Oct 30 20:07:35.194289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.194257 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-tmp-dir" pod "4f940223-245c-4132-880c-dd2f75dfe9cf" (UID: "4f940223-245c-4132-880c-dd2f75dfe9cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:35.194514 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.194488 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-home" pod "4f940223-245c-4132-880c-dd2f75dfe9cf" (UID: "4f940223-245c-4132-880c-dd2f75dfe9cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:35.195792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.195763 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-model-cache" pod "4f940223-245c-4132-880c-dd2f75dfe9cf" (UID: "4f940223-245c-4132-880c-dd2f75dfe9cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:35.196492 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.196459 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f940223-245c-4132-880c-dd2f75dfe9cf-tls-certs" pod "4f940223-245c-4132-880c-dd2f75dfe9cf" (UID: "4f940223-245c-4132-880c-dd2f75dfe9cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:35.196629 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.196604 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f940223-245c-4132-880c-dd2f75dfe9cf-kube-api-access-74nvh" pod "4f940223-245c-4132-880c-dd2f75dfe9cf" (UID: "4f940223-245c-4132-880c-dd2f75dfe9cf"). InnerVolumeSpecName "kube-api-access-74nvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:35.198295 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.198271 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-dshm" pod "4f940223-245c-4132-880c-dd2f75dfe9cf" (UID: "4f940223-245c-4132-880c-dd2f75dfe9cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:35.199258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.199229 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:35.199258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.199256 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:35.199414 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.199269 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:35.199414 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.199282 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74nvh\" (UniqueName: \"kubernetes.io/projected/4f940223-245c-4132-880c-dd2f75dfe9cf-kube-api-access-74nvh\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:35.199414 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.199295 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940223-245c-4132-880c-dd2f75dfe9cf-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:35.199414 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.199308 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940223-245c-4132-880c-dd2f75dfe9cf-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:35.280984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.280952 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:35.381426 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.381395 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" Oct 30 20:07:35.381878 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.381428 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb" event={"ID":"4f940223-245c-4132-880c-dd2f75dfe9cf","Type":"ContainerDied","Data":"62ac115ac0ca459bdadeddf629700e3e6b9dfe01a86a28e3b2f0bc561b47ab60"} Oct 30 20:07:35.381878 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.381475 6411 scope.go:122] "RemoveContainer" containerID="9d53bd631d2dc7d744fa5fa9e7a035691ac46c4e95b8d708508f98f8d1a5928e" Oct 30 20:07:35.452293 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.452210 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb"] Oct 30 20:07:35.464523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.464495 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-626870/distinct-w5-626870-kserve-85b5597ddf-w8mqb"] Oct 30 20:07:35.734194 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:35.734120 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" path="/var/lib/kubelet/pods/4f940223-245c-4132-880c-dd2f75dfe9cf/volumes" Oct 30 20:07:36.782914 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:36.782876 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:38.325067 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:38.325008 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:39.996652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:39.996618 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl"] Oct 30 20:07:40.008323 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:40.008293 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r"] Oct 30 20:07:40.027163 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:40.027138 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx"] Oct 30 20:07:40.137749 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:40.137703 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c"] Oct 30 20:07:40.182442 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:40.182403 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:40.385101 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:40.385061 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:41.769536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:41.769504 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:42.330341 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:42.330304 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:42.696066 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:42.696031 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" podUID="5c095837-e094-43c1-aa14-5715094b2bbb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 20:07:43.893251 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:43.893215 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r"] Oct 30 20:07:43.893721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:43.893487 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" podUID="b39478e6-123e-48ad-b7f4-d48da825d8ee" containerName="maas-api" containerID="cri-o://4b224ced077af21dae5dcd1bf9412a544ba0b824a6c0414cc844bf9c02e5d960" gracePeriod=30 Oct 30 20:07:43.936978 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:43.936934 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:44.366069 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.365766 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:44.411344 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.411048 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km"] Oct 30 20:07:44.411344 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.411285 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" podUID="fda073ed-6c46-42ef-a592-c99c980f8885" containerName="maas-api" containerID="cri-o://f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137" gracePeriod=30 Oct 30 20:07:44.463544 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.463378 6411 generic.go:356] "Generic (PLEG): container finished" podID="b39478e6-123e-48ad-b7f4-d48da825d8ee" containerID="4b224ced077af21dae5dcd1bf9412a544ba0b824a6c0414cc844bf9c02e5d960" exitCode=0 Oct 30 20:07:44.463544 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.463456 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" event={"ID":"b39478e6-123e-48ad-b7f4-d48da825d8ee","Type":"ContainerDied","Data":"4b224ced077af21dae5dcd1bf9412a544ba0b824a6c0414cc844bf9c02e5d960"} Oct 30 20:07:44.641288 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.633577 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:44.696677 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.696370 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:07:44.790976 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.790626 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-metrics-certs\") pod \"b39478e6-123e-48ad-b7f4-d48da825d8ee\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " Oct 30 20:07:44.790976 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.790763 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls\") pod \"b39478e6-123e-48ad-b7f4-d48da825d8ee\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " Oct 30 20:07:44.791202 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.791027 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b39478e6-123e-48ad-b7f4-d48da825d8ee-kube-api-access-ctqvk\" (UniqueName: \"kubernetes.io/projected/b39478e6-123e-48ad-b7f4-d48da825d8ee-kube-api-access-ctqvk\") pod \"b39478e6-123e-48ad-b7f4-d48da825d8ee\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " Oct 30 20:07:44.791202 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.791100 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b39478e6-123e-48ad-b7f4-d48da825d8ee-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39478e6-123e-48ad-b7f4-d48da825d8ee-openshift-service-ca\") pod \"b39478e6-123e-48ad-b7f4-d48da825d8ee\" (UID: \"b39478e6-123e-48ad-b7f4-d48da825d8ee\") " Oct 30 20:07:44.806260 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.792159 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b39478e6-123e-48ad-b7f4-d48da825d8ee-openshift-service-ca" pod "b39478e6-123e-48ad-b7f4-d48da825d8ee" (UID: "b39478e6-123e-48ad-b7f4-d48da825d8ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:07:44.809485 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.809431 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-metrics-certs" pod "b39478e6-123e-48ad-b7f4-d48da825d8ee" (UID: "b39478e6-123e-48ad-b7f4-d48da825d8ee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:44.820281 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.817045 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls" pod "b39478e6-123e-48ad-b7f4-d48da825d8ee" (UID: "b39478e6-123e-48ad-b7f4-d48da825d8ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:44.820281 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.817068 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39478e6-123e-48ad-b7f4-d48da825d8ee-kube-api-access-ctqvk" pod "b39478e6-123e-48ad-b7f4-d48da825d8ee" (UID: "b39478e6-123e-48ad-b7f4-d48da825d8ee"). InnerVolumeSpecName "kube-api-access-ctqvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:44.885499 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.885353 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:44.892951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.892856 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctqvk\" (UniqueName: \"kubernetes.io/projected/b39478e6-123e-48ad-b7f4-d48da825d8ee-kube-api-access-ctqvk\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:44.892951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.892887 6411 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39478e6-123e-48ad-b7f4-d48da825d8ee-openshift-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:44.892951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.892919 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:44.892951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.892929 6411 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39478e6-123e-48ad-b7f4-d48da825d8ee-maas-api-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:44.973450 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:44.973423 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:07:45.152541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.143312 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fda073ed-6c46-42ef-a592-c99c980f8885-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda073ed-6c46-42ef-a592-c99c980f8885-openshift-service-ca\") pod \"fda073ed-6c46-42ef-a592-c99c980f8885\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " Oct 30 20:07:45.152541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.143660 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fda073ed-6c46-42ef-a592-c99c980f8885-openshift-service-ca" pod "fda073ed-6c46-42ef-a592-c99c980f8885" (UID: "fda073ed-6c46-42ef-a592-c99c980f8885"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:07:45.152541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.143858 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-maas-api-tls\") pod \"fda073ed-6c46-42ef-a592-c99c980f8885\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " Oct 30 20:07:45.152541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.144206 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-metrics-certs\") pod \"fda073ed-6c46-42ef-a592-c99c980f8885\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " Oct 30 20:07:45.152541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.144625 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fda073ed-6c46-42ef-a592-c99c980f8885-kube-api-access-8tgtx\" (UniqueName: \"kubernetes.io/projected/fda073ed-6c46-42ef-a592-c99c980f8885-kube-api-access-8tgtx\") pod \"fda073ed-6c46-42ef-a592-c99c980f8885\" (UID: \"fda073ed-6c46-42ef-a592-c99c980f8885\") " Oct 30 20:07:45.152541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.145360 6411 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda073ed-6c46-42ef-a592-c99c980f8885-openshift-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:45.153400 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.153352 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-maas-api-tls" pod "fda073ed-6c46-42ef-a592-c99c980f8885" (UID: "fda073ed-6c46-42ef-a592-c99c980f8885"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:45.153592 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.153556 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-metrics-certs" pod "fda073ed-6c46-42ef-a592-c99c980f8885" (UID: "fda073ed-6c46-42ef-a592-c99c980f8885"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:45.154307 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.154285 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda073ed-6c46-42ef-a592-c99c980f8885-kube-api-access-8tgtx" pod "fda073ed-6c46-42ef-a592-c99c980f8885" (UID: "fda073ed-6c46-42ef-a592-c99c980f8885"). InnerVolumeSpecName "kube-api-access-8tgtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:45.203470 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.203435 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:45.247001 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.246967 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tgtx\" (UniqueName: \"kubernetes.io/projected/fda073ed-6c46-42ef-a592-c99c980f8885-kube-api-access-8tgtx\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:45.247001 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.247001 6411 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-maas-api-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:45.247262 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.247019 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fda073ed-6c46-42ef-a592-c99c980f8885-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:45.404207 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.404094 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:45.475046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.475009 6411 generic.go:356] "Generic (PLEG): container finished" podID="fda073ed-6c46-42ef-a592-c99c980f8885" containerID="f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137" exitCode=0 Oct 30 20:07:45.475258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.475070 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" Oct 30 20:07:45.475258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.475220 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" event={"ID":"fda073ed-6c46-42ef-a592-c99c980f8885","Type":"ContainerDied","Data":"f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137"} Oct 30 20:07:45.475381 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.475270 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km" event={"ID":"fda073ed-6c46-42ef-a592-c99c980f8885","Type":"ContainerDied","Data":"5bc226a5220dc5243c0da239f56868782f77f7d786a8c84fd8617cf4f58beab3"} Oct 30 20:07:45.475381 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.475300 6411 scope.go:122] "RemoveContainer" containerID="f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137" Oct 30 20:07:45.477253 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.477227 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" event={"ID":"b39478e6-123e-48ad-b7f4-d48da825d8ee","Type":"ContainerDied","Data":"3429d30b6124dec044317fcfcd01e4d4c880c3b37e49a0f3573c75af998452d8"} Oct 30 20:07:45.477514 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.477496 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r" Oct 30 20:07:45.499658 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.499107 6411 scope.go:122] "RemoveContainer" containerID="f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137" Oct 30 20:07:45.500470 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:07:45.500442 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137\": container with ID starting with f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137 not found: ID does not exist" containerID="f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137" Oct 30 20:07:45.500545 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.500484 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137"} err="failed to get container status \"f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137\": rpc error: code = NotFound desc = could not find container \"f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137\": container with ID starting with f4cb6ee5a8ec970fe432a49b77182a2295d967f520f491a1c2526eed2bafe137 not found: ID does not exist" Oct 30 20:07:45.500545 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.500509 6411 scope.go:122] "RemoveContainer" containerID="4b224ced077af21dae5dcd1bf9412a544ba0b824a6c0414cc844bf9c02e5d960" Oct 30 20:07:45.600398 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.600355 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km"] Oct 30 20:07:45.605116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.605085 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-626870-79795b9b56-hv9km"] Oct 30 20:07:45.628949 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.628889 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r"] Oct 30 20:07:45.634825 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.634792 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-f1f016-79ccc8985b-r2r4r"] Oct 30 20:07:45.736867 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.736791 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b39478e6-123e-48ad-b7f4-d48da825d8ee" path="/var/lib/kubelet/pods/b39478e6-123e-48ad-b7f4-d48da825d8ee/volumes" Oct 30 20:07:45.737306 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:45.737290 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fda073ed-6c46-42ef-a592-c99c980f8885" path="/var/lib/kubelet/pods/fda073ed-6c46-42ef-a592-c99c980f8885/volumes" Oct 30 20:07:46.039078 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:46.039002 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:46.355156 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:46.355069 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:46.933046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:46.932804 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:46.993031 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:46.988889 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk"] Oct 30 20:07:47.287652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.279931 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2"] Oct 30 20:07:47.287652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.280224 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" podUID="4a92199c-6d6c-4020-b877-8e585a87cd28" containerName="maas-api" containerID="cri-o://2d02b1184980bfd8e5441d9424849aaf0a39412c9a68b39d054ee2ed3219d2c6" gracePeriod=30 Oct 30 20:07:47.319176 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.319144 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:47.504446 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.504406 6411 generic.go:356] "Generic (PLEG): container finished" podID="4a92199c-6d6c-4020-b877-8e585a87cd28" containerID="2d02b1184980bfd8e5441d9424849aaf0a39412c9a68b39d054ee2ed3219d2c6" exitCode=0 Oct 30 20:07:47.504635 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.504524 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" event={"ID":"4a92199c-6d6c-4020-b877-8e585a87cd28","Type":"ContainerDied","Data":"2d02b1184980bfd8e5441d9424849aaf0a39412c9a68b39d054ee2ed3219d2c6"} Oct 30 20:07:47.511815 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.510462 6411 generic.go:356] "Generic (PLEG): container finished" podID="5c095837-e094-43c1-aa14-5715094b2bbb" containerID="57a6fe4b1a817535b7bf93e49682a4253f5405cb800c8a97c37695c1df515211" exitCode=0 Oct 30 20:07:47.511815 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.510554 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" event={"ID":"5c095837-e094-43c1-aa14-5715094b2bbb","Type":"ContainerDied","Data":"57a6fe4b1a817535b7bf93e49682a4253f5405cb800c8a97c37695c1df515211"} Oct 30 20:07:47.574992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.574340 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:47.725404 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.725162 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:07:47.749982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.749948 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:07:47.826070 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.808403 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a92199c-6d6c-4020-b877-8e585a87cd28-kube-api-access-5w4wf\" (UniqueName: \"kubernetes.io/projected/4a92199c-6d6c-4020-b877-8e585a87cd28-kube-api-access-5w4wf\") pod \"4a92199c-6d6c-4020-b877-8e585a87cd28\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " Oct 30 20:07:47.826070 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.808520 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a92199c-6d6c-4020-b877-8e585a87cd28-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92199c-6d6c-4020-b877-8e585a87cd28-openshift-service-ca\") pod \"4a92199c-6d6c-4020-b877-8e585a87cd28\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " Oct 30 20:07:47.826070 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.808560 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c095837-e094-43c1-aa14-5715094b2bbb-kube-api-access-wh5lw\" (UniqueName: \"kubernetes.io/projected/5c095837-e094-43c1-aa14-5715094b2bbb-kube-api-access-wh5lw\") pod \"5c095837-e094-43c1-aa14-5715094b2bbb\" (UID: \"5c095837-e094-43c1-aa14-5715094b2bbb\") " Oct 30 20:07:47.826070 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.808622 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c095837-e094-43c1-aa14-5715094b2bbb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c095837-e094-43c1-aa14-5715094b2bbb-plugins-config-volume\") pod \"5c095837-e094-43c1-aa14-5715094b2bbb\" (UID: \"5c095837-e094-43c1-aa14-5715094b2bbb\") " Oct 30 20:07:47.826070 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.808749 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls\") pod \"4a92199c-6d6c-4020-b877-8e585a87cd28\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " Oct 30 20:07:47.826070 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.808861 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-metrics-certs\") pod \"4a92199c-6d6c-4020-b877-8e585a87cd28\" (UID: \"4a92199c-6d6c-4020-b877-8e585a87cd28\") " Oct 30 20:07:47.850952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.850428 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a92199c-6d6c-4020-b877-8e585a87cd28-openshift-service-ca" pod "4a92199c-6d6c-4020-b877-8e585a87cd28" (UID: "4a92199c-6d6c-4020-b877-8e585a87cd28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:07:47.850952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.850858 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c095837-e094-43c1-aa14-5715094b2bbb-plugins-config-volume" pod "5c095837-e094-43c1-aa14-5715094b2bbb" (UID: "5c095837-e094-43c1-aa14-5715094b2bbb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:07:47.879182 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.879145 6411 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92199c-6d6c-4020-b877-8e585a87cd28-openshift-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:47.879182 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.879182 6411 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c095837-e094-43c1-aa14-5715094b2bbb-plugins-config-volume\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:47.932235 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.932172 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a92199c-6d6c-4020-b877-8e585a87cd28-kube-api-access-5w4wf" pod "4a92199c-6d6c-4020-b877-8e585a87cd28" (UID: "4a92199c-6d6c-4020-b877-8e585a87cd28"). InnerVolumeSpecName "kube-api-access-5w4wf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:47.956231 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.942108 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c095837-e094-43c1-aa14-5715094b2bbb-kube-api-access-wh5lw" pod "5c095837-e094-43c1-aa14-5715094b2bbb" (UID: "5c095837-e094-43c1-aa14-5715094b2bbb"). InnerVolumeSpecName "kube-api-access-wh5lw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:48.005923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:47.999973 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w4wf\" (UniqueName: \"kubernetes.io/projected/4a92199c-6d6c-4020-b877-8e585a87cd28-kube-api-access-5w4wf\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:48.005923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.000007 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh5lw\" (UniqueName: \"kubernetes.io/projected/5c095837-e094-43c1-aa14-5715094b2bbb-kube-api-access-wh5lw\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:48.057182 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.057100 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-metrics-certs" pod "4a92199c-6d6c-4020-b877-8e585a87cd28" (UID: "4a92199c-6d6c-4020-b877-8e585a87cd28"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:48.092008 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.088180 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls" pod "4a92199c-6d6c-4020-b877-8e585a87cd28" (UID: "4a92199c-6d6c-4020-b877-8e585a87cd28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:48.127104 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.113037 6411 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-maas-api-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:48.127104 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.113070 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92199c-6d6c-4020-b877-8e585a87cd28-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:48.134770 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.134696 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:48.323663 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.323594 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:48.542563 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.531769 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:48.542563 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.541048 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" event={"ID":"4a92199c-6d6c-4020-b877-8e585a87cd28","Type":"ContainerDied","Data":"e0d12e401e1411ea39ae4bccc5350c154ebe34da13e7f1652a5b87b68ae49f30"} Oct 30 20:07:48.542563 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.541119 6411 scope.go:122] "RemoveContainer" containerID="2d02b1184980bfd8e5441d9424849aaf0a39412c9a68b39d054ee2ed3219d2c6" Oct 30 20:07:48.542563 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.541408 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2" Oct 30 20:07:48.554923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.549380 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" event={"ID":"5c095837-e094-43c1-aa14-5715094b2bbb","Type":"ContainerDied","Data":"e2b5cd1d706f62551a4da8290b9eedbe663536daf9a9c4acf93abf5b50c96c2a"} Oct 30 20:07:48.554923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.549489 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c" Oct 30 20:07:48.586279 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.586250 6411 scope.go:122] "RemoveContainer" containerID="57a6fe4b1a817535b7bf93e49682a4253f5405cb800c8a97c37695c1df515211" Oct 30 20:07:48.636497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.636465 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c"] Oct 30 20:07:48.641302 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.641277 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-626870-85df8fcd5c-px29c"] Oct 30 20:07:48.682835 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.682798 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2"] Oct 30 20:07:48.687508 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.687484 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-912f24-54467cdd9f-rxkp2"] Oct 30 20:07:48.747924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:48.745706 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:49.028270 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:49.028231 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:49.749555 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:49.749514 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a92199c-6d6c-4020-b877-8e585a87cd28" path="/var/lib/kubelet/pods/4a92199c-6d6c-4020-b877-8e585a87cd28/volumes" Oct 30 20:07:49.750967 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:49.750941 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c095837-e094-43c1-aa14-5715094b2bbb" path="/var/lib/kubelet/pods/5c095837-e094-43c1-aa14-5715094b2bbb/volumes" Oct 30 20:07:51.096923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:51.091002 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:51.251025 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:51.250980 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:51.333052 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:51.333006 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:51.840586 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:51.840006 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:51.993303 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:51.993111 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" podUID="aeddd424-8ad6-40dd-aaec-68195c43415e" containerName="payload-pre-processing" containerID="cri-o://0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569" gracePeriod=25 Oct 30 20:07:52.110984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.098221 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:52.110984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.098932 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:07:52.187930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.186989 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:52.254943 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.253101 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:52.254943 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.253780 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:07:52.320965 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.302366 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:52.424032 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.346999 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:52.424032 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.348989 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:07:52.424032 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.349774 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:07:52.877982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.855309 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:52.877982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:52.856796 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:07:53.046923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.046839 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:53.101923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.101726 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:53.310015 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.309969 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:53.312796 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.312759 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:53.314073 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.314053 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:07:53.363924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.353221 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:53.404932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.369195 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:53.404932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.401852 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:53.404932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.388649 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:07:53.718982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.718889 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:53.865370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:53.865329 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:54.326601 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:54.326541 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:07:54.406359 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:54.352065 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:07:54.447146 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:54.447084 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:54.484926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:54.449948 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:07:54.484926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:54.452135 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:54.984849 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:07:54.984810 6411 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.247s" Oct 30 20:07:54.985382 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:54.985359 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:55.065335 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.052275 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:55.077954 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.066398 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" containerID="cri-o://dd8f3397940dffb3b0a7e42f0c63788cc82a3cf9dbdd9f47c2d980f42d30ab9b" gracePeriod=45 Oct 30 20:07:55.077954 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.066634 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" containerID="cri-o://7ca45219e56ddcd5ab7bebecaad1f78624e2bf346e046ca2a14c258e49436792" gracePeriod=45 Oct 30 20:07:55.198950 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.189223 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x"] Oct 30 20:07:55.301258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.265226 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" containerID="cri-o://65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350" gracePeriod=30 Oct 30 20:07:55.340927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.318166 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" containerID="cri-o://ae629cf7663866dc4d7c0e560ec4b933ca15ebcedb145d6856f4dd5317962f30" gracePeriod=45 Oct 30 20:07:55.399071 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.330691 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5"] Oct 30 20:07:55.399495 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.399461 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" containerID="cri-o://e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f" gracePeriod=30 Oct 30 20:07:55.399744 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.209120 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" containerID="cri-o://52773eb19098882b93d93810587ada6953ff99a9ad679355e4634305eabafe74" gracePeriod=45 Oct 30 20:07:55.420602 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.418465 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:55.433933 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.426455 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:55.442965 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.435233 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded" Oct 30 20:07:55.535271 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.478609 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:55.578126 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.578031 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": read tcp 10.133.0.2:48248->10.133.0.64:8000: read: connection reset by peer" Oct 30 20:07:55.588930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.530545 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:55.595010 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.569190 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:55.595010 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.480925 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:55.912200 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.912150 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd"] Oct 30 20:07:55.912774 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:55.912724 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" containerID="cri-o://31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3" gracePeriod=30 Oct 30 20:07:56.064075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.053944 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:56.107936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.107411 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": dial tcp 10.133.0.54:8000: connect: connection refused" Oct 30 20:07:56.138041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.137937 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:56.289625 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.283092 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": dial tcp 10.133.0.64:8000: connect: connection refused" Oct 30 20:07:56.497980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.443677 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:56.497980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.443003 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:56.497980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.473520 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:56.497980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.487055 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g"] Oct 30 20:07:56.497980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.487657 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" containerID="cri-o://f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132" gracePeriod=30 Oct 30 20:07:56.561271 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.561187 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": dial tcp 10.133.0.62:8000: connect: connection refused" Oct 30 20:07:56.593934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.592955 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded" Oct 30 20:07:56.593934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.593560 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": dial tcp 10.133.0.55:8000: connect: connection refused" Oct 30 20:07:56.619937 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.603263 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:56.632536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.627134 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn"] Oct 30 20:07:56.632536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.628460 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" containerID="cri-o://9f126c64cd60a90eecc036674719338cea7dc22b6ddf5253a04f056863d558db" gracePeriod=30 Oct 30 20:07:56.755345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.747993 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx"] Oct 30 20:07:56.755345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.748231 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" containerID="cri-o://d1d48d91f9feac5515189b72f794deed33786fb80e12767afdc701f4fa62916d" gracePeriod=30 Oct 30 20:07:56.865935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:56.844071 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": dial tcp 10.133.0.54:8000: connect: connection refused" Oct 30 20:07:57.022046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.021629 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:07:57.037039 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.037014 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:07:57.205536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.180675 6411 generic.go:356] "Generic (PLEG): container finished" podID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerID="ae629cf7663866dc4d7c0e560ec4b933ca15ebcedb145d6856f4dd5317962f30" exitCode=0 Oct 30 20:07:57.205536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.180802 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" event={"ID":"97a27bca-4f25-4da5-ad54-e4dab81fe99b","Type":"ContainerDied","Data":"ae629cf7663866dc4d7c0e560ec4b933ca15ebcedb145d6856f4dd5317962f30"} Oct 30 20:07:57.205536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.183116 6411 generic.go:356] "Generic (PLEG): container finished" podID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerID="52773eb19098882b93d93810587ada6953ff99a9ad679355e4634305eabafe74" exitCode=0 Oct 30 20:07:57.205536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.183235 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" event={"ID":"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca","Type":"ContainerDied","Data":"52773eb19098882b93d93810587ada6953ff99a9ad679355e4634305eabafe74"} Oct 30 20:07:57.205536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.202374 6411 generic.go:356] "Generic (PLEG): container finished" podID="981c36db-e723-499d-802f-cbfb370addee" containerID="7ca45219e56ddcd5ab7bebecaad1f78624e2bf346e046ca2a14c258e49436792" exitCode=0 Oct 30 20:07:57.205536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.202875 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" event={"ID":"981c36db-e723-499d-802f-cbfb370addee","Type":"ContainerDied","Data":"7ca45219e56ddcd5ab7bebecaad1f78624e2bf346e046ca2a14c258e49436792"} Oct 30 20:07:57.205536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.205133 6411 generic.go:356] "Generic (PLEG): container finished" podID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerID="dd8f3397940dffb3b0a7e42f0c63788cc82a3cf9dbdd9f47c2d980f42d30ab9b" exitCode=0 Oct 30 20:07:57.205536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.205249 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" event={"ID":"00f5fddd-2204-412a-bee6-ddd2dbe13213","Type":"ContainerDied","Data":"dd8f3397940dffb3b0a7e42f0c63788cc82a3cf9dbdd9f47c2d980f42d30ab9b"} Oct 30 20:07:57.219945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.217849 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:07:57.324969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.324638 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": dial tcp 10.133.0.55:8000: connect: connection refused" Oct 30 20:07:57.358095 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.358057 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-home\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-home\") pod \"00f5fddd-2204-412a-bee6-ddd2dbe13213\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " Oct 30 20:07:57.358282 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.358143 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00f5fddd-2204-412a-bee6-ddd2dbe13213-kube-api-access-wbw6x\" (UniqueName: \"kubernetes.io/projected/00f5fddd-2204-412a-bee6-ddd2dbe13213-kube-api-access-wbw6x\") pod \"00f5fddd-2204-412a-bee6-ddd2dbe13213\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " Oct 30 20:07:57.358282 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.358203 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00f5fddd-2204-412a-bee6-ddd2dbe13213-tls-certs\" (UniqueName: \"kubernetes.io/secret/00f5fddd-2204-412a-bee6-ddd2dbe13213-tls-certs\") pod \"00f5fddd-2204-412a-bee6-ddd2dbe13213\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " Oct 30 20:07:57.358403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.358311 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-dshm\") pod \"00f5fddd-2204-412a-bee6-ddd2dbe13213\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " Oct 30 20:07:57.358403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.358356 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-model-cache\") pod \"00f5fddd-2204-412a-bee6-ddd2dbe13213\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " Oct 30 20:07:57.358546 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.358527 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-tmp-dir\") pod \"00f5fddd-2204-412a-bee6-ddd2dbe13213\" (UID: \"00f5fddd-2204-412a-bee6-ddd2dbe13213\") " Oct 30 20:07:57.359830 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.359322 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-tmp-dir" pod "00f5fddd-2204-412a-bee6-ddd2dbe13213" (UID: "00f5fddd-2204-412a-bee6-ddd2dbe13213"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.359830 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.359530 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-home" pod "00f5fddd-2204-412a-bee6-ddd2dbe13213" (UID: "00f5fddd-2204-412a-bee6-ddd2dbe13213"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.363009 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.362966 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-model-cache" pod "00f5fddd-2204-412a-bee6-ddd2dbe13213" (UID: "00f5fddd-2204-412a-bee6-ddd2dbe13213"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.363205 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.363011 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f5fddd-2204-412a-bee6-ddd2dbe13213-kube-api-access-wbw6x" pod "00f5fddd-2204-412a-bee6-ddd2dbe13213" (UID: "00f5fddd-2204-412a-bee6-ddd2dbe13213"). InnerVolumeSpecName "kube-api-access-wbw6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:57.365662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.365630 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-dshm" pod "00f5fddd-2204-412a-bee6-ddd2dbe13213" (UID: "00f5fddd-2204-412a-bee6-ddd2dbe13213"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.365753 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.365725 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f5fddd-2204-412a-bee6-ddd2dbe13213-tls-certs" pod "00f5fddd-2204-412a-bee6-ddd2dbe13213" (UID: "00f5fddd-2204-412a-bee6-ddd2dbe13213"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:57.438953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.435061 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:57.467458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.460726 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.467458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.460760 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbw6x\" (UniqueName: \"kubernetes.io/projected/00f5fddd-2204-412a-bee6-ddd2dbe13213-kube-api-access-wbw6x\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.467458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.460776 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f5fddd-2204-412a-bee6-ddd2dbe13213-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.467458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.460789 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.467458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.460802 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.467458 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.460812 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f5fddd-2204-412a-bee6-ddd2dbe13213-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.568734 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.568688 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-tmp-dir\") pod \"981c36db-e723-499d-802f-cbfb370addee\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " Oct 30 20:07:57.569473 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.569448 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/981c36db-e723-499d-802f-cbfb370addee-tls-certs\" (UniqueName: \"kubernetes.io/secret/981c36db-e723-499d-802f-cbfb370addee-tls-certs\") pod \"981c36db-e723-499d-802f-cbfb370addee\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " Oct 30 20:07:57.569654 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.569640 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-home\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-home\") pod \"981c36db-e723-499d-802f-cbfb370addee\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " Oct 30 20:07:57.569782 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.569769 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-dshm\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-dshm\") pod \"981c36db-e723-499d-802f-cbfb370addee\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " Oct 30 20:07:57.569952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.569937 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/981c36db-e723-499d-802f-cbfb370addee-kube-api-access-phrl6\" (UniqueName: \"kubernetes.io/projected/981c36db-e723-499d-802f-cbfb370addee-kube-api-access-phrl6\") pod \"981c36db-e723-499d-802f-cbfb370addee\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " Oct 30 20:07:57.570116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.570100 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-model-cache\") pod \"981c36db-e723-499d-802f-cbfb370addee\" (UID: \"981c36db-e723-499d-802f-cbfb370addee\") " Oct 30 20:07:57.581417 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.581333 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-dshm" pod "981c36db-e723-499d-802f-cbfb370addee" (UID: "981c36db-e723-499d-802f-cbfb370addee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.598559 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.598520 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-model-cache" pod "981c36db-e723-499d-802f-cbfb370addee" (UID: "981c36db-e723-499d-802f-cbfb370addee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.598721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.598669 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-home" pod "981c36db-e723-499d-802f-cbfb370addee" (UID: "981c36db-e723-499d-802f-cbfb370addee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.598721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.598711 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-tmp-dir" pod "981c36db-e723-499d-802f-cbfb370addee" (UID: "981c36db-e723-499d-802f-cbfb370addee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.614025 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.613965 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981c36db-e723-499d-802f-cbfb370addee-tls-certs" pod "981c36db-e723-499d-802f-cbfb370addee" (UID: "981c36db-e723-499d-802f-cbfb370addee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:57.632955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.629149 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981c36db-e723-499d-802f-cbfb370addee-kube-api-access-phrl6" pod "981c36db-e723-499d-802f-cbfb370addee" (UID: "981c36db-e723-499d-802f-cbfb370addee"). InnerVolumeSpecName "kube-api-access-phrl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:57.638414 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.638218 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:57.672324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.672146 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981c36db-e723-499d-802f-cbfb370addee-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.672324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.672176 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.672324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.672188 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.672324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.672201 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phrl6\" (UniqueName: \"kubernetes.io/projected/981c36db-e723-499d-802f-cbfb370addee-kube-api-access-phrl6\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.672324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.672214 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.672324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.672227 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981c36db-e723-499d-802f-cbfb370addee-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.724615 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.713423 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:07:57.748819 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.748783 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" podUID="aeddd424-8ad6-40dd-aaec-68195c43415e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 20:07:57.765031 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.764996 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:07:57.780845 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.778647 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97a27bca-4f25-4da5-ad54-e4dab81fe99b-kube-api-access-jt8j5\" (UniqueName: \"kubernetes.io/projected/97a27bca-4f25-4da5-ad54-e4dab81fe99b-kube-api-access-jt8j5\") pod \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " Oct 30 20:07:57.780845 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.778738 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-home\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-home\") pod \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " Oct 30 20:07:57.780845 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.778767 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-dshm\") pod \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " Oct 30 20:07:57.780845 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.778800 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tls-certs\") pod \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " Oct 30 20:07:57.780845 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.778860 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-model-cache\") pod \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " Oct 30 20:07:57.781263 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.781064 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-dshm\") pod \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " Oct 30 20:07:57.781263 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.781136 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-model-cache\") pod \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " Oct 30 20:07:57.781368 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.781266 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-kube-api-access-cvnv4\" (UniqueName: \"kubernetes.io/projected/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-kube-api-access-cvnv4\") pod \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " Oct 30 20:07:57.781368 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.781314 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tls-certs\" (UniqueName: \"kubernetes.io/secret/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tls-certs\") pod \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " Oct 30 20:07:57.781482 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.781405 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tmp-dir\") pod \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\" (UID: \"97a27bca-4f25-4da5-ad54-e4dab81fe99b\") " Oct 30 20:07:57.781482 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.781472 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-home\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-home\") pod \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " Oct 30 20:07:57.781624 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.781592 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tmp-dir\") pod \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\" (UID: \"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca\") " Oct 30 20:07:57.783044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.783013 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tmp-dir" pod "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" (UID: "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.783516 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.783486 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-dshm" pod "97a27bca-4f25-4da5-ad54-e4dab81fe99b" (UID: "97a27bca-4f25-4da5-ad54-e4dab81fe99b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.786720 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.785007 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tmp-dir" pod "97a27bca-4f25-4da5-ad54-e4dab81fe99b" (UID: "97a27bca-4f25-4da5-ad54-e4dab81fe99b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.788010 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.787986 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tls-certs" pod "97a27bca-4f25-4da5-ad54-e4dab81fe99b" (UID: "97a27bca-4f25-4da5-ad54-e4dab81fe99b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:57.788796 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.788606 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tls-certs" pod "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" (UID: "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:07:57.788998 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.788978 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-home" pod "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" (UID: "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.790614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.790368 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97a27bca-4f25-4da5-ad54-e4dab81fe99b-kube-api-access-jt8j5" pod "97a27bca-4f25-4da5-ad54-e4dab81fe99b" (UID: "97a27bca-4f25-4da5-ad54-e4dab81fe99b"). InnerVolumeSpecName "kube-api-access-jt8j5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:57.790614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.790370 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-home" pod "97a27bca-4f25-4da5-ad54-e4dab81fe99b" (UID: "97a27bca-4f25-4da5-ad54-e4dab81fe99b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.790614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.790407 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-kube-api-access-cvnv4" pod "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" (UID: "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca"). InnerVolumeSpecName "kube-api-access-cvnv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:07:57.790810 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.790631 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-model-cache" pod "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" (UID: "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.792588 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.790986 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-model-cache" pod "97a27bca-4f25-4da5-ad54-e4dab81fe99b" (UID: "97a27bca-4f25-4da5-ad54-e4dab81fe99b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.803944 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.803099 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-dshm" pod "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" (UID: "9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:07:57.861433 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.861350 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:57.887325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887269 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887317 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt8j5\" (UniqueName: \"kubernetes.io/projected/97a27bca-4f25-4da5-ad54-e4dab81fe99b-kube-api-access-jt8j5\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887334 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887347 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887359 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887371 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887382 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887394 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887406 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvnv4\" (UniqueName: \"kubernetes.io/projected/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-kube-api-access-cvnv4\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887418 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887431 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97a27bca-4f25-4da5-ad54-e4dab81fe99b-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:57.887743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:57.887442 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:07:58.217384 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.217133 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" event={"ID":"981c36db-e723-499d-802f-cbfb370addee","Type":"ContainerDied","Data":"3d34b86b4e2433a6359ff59e03bd8fd527ff6a0828de235cc8a008c44008e89f"} Oct 30 20:07:58.217384 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.217223 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" Oct 30 20:07:58.217384 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.217276 6411 scope.go:122] "RemoveContainer" containerID="7ca45219e56ddcd5ab7bebecaad1f78624e2bf346e046ca2a14c258e49436792" Oct 30 20:07:58.221648 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.221626 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" Oct 30 20:07:58.223189 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.223030 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl" event={"ID":"00f5fddd-2204-412a-bee6-ddd2dbe13213","Type":"ContainerDied","Data":"c879b88f2aa600096c27e4975a38d8b4b208efe5ddd4e84b766ef3a894eedb1a"} Oct 30 20:07:58.225269 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.225242 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" event={"ID":"97a27bca-4f25-4da5-ad54-e4dab81fe99b","Type":"ContainerDied","Data":"58d6f0eaf68055429bc34958eaa9b7677d97e0b60b80e6e4262c79edadc92321"} Oct 30 20:07:58.225397 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.225318 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c" Oct 30 20:07:58.232062 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.232039 6411 scope.go:122] "RemoveContainer" containerID="dd8f3397940dffb3b0a7e42f0c63788cc82a3cf9dbdd9f47c2d980f42d30ab9b" Oct 30 20:07:58.232338 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.232315 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" event={"ID":"9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca","Type":"ContainerDied","Data":"dd1acac5a1cf2d3fe0d54b54f5cf2a919a48504016dd8bc58eb4b49b1e871458"} Oct 30 20:07:58.232448 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.232413 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx" Oct 30 20:07:58.247889 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.247867 6411 scope.go:122] "RemoveContainer" containerID="ae629cf7663866dc4d7c0e560ec4b933ca15ebcedb145d6856f4dd5317962f30" Oct 30 20:07:58.268135 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.268113 6411 scope.go:122] "RemoveContainer" containerID="52773eb19098882b93d93810587ada6953ff99a9ad679355e4634305eabafe74" Oct 30 20:07:58.296299 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.296257 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r"] Oct 30 20:07:58.306029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.306001 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r"] Oct 30 20:07:58.323387 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.323353 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-62f4a1/distinct-w0-62f4a1-kserve-7646b99569-7cq8r" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:07:58.347040 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.347007 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl"] Oct 30 20:07:58.354995 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.354960 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/distinct-2-w0-62f4a1-kserve-68fdf56ff8-zlpnl"] Oct 30 20:07:58.411846 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.411797 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c"] Oct 30 20:07:58.418446 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.418417 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-5f10d2/simulator-w1-5f10d2-kserve-fd4877896-d765c"] Oct 30 20:07:58.455473 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.455379 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx"] Oct 30 20:07:58.458124 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.458094 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-62f4a1/premium-w0-62f4a1-kserve-757d46bc8-fwgmx"] Oct 30 20:07:58.809838 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:58.809743 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:59.055132 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.055087 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:59.279805 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.279770 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv"] Oct 30 20:07:59.280114 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.280086 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" containerID="cri-o://d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f" gracePeriod=30 Oct 30 20:07:59.282968 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.282943 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd"] Oct 30 20:07:59.283159 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.283135 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" containerID="cri-o://7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9" gracePeriod=30 Oct 30 20:07:59.318539 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.318502 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf"] Oct 30 20:07:59.318754 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.318731 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" containerID="cri-o://c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f" gracePeriod=30 Oct 30 20:07:59.452807 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.452614 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:07:59.733193 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.733162 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" path="/var/lib/kubelet/pods/00f5fddd-2204-412a-bee6-ddd2dbe13213/volumes" Oct 30 20:07:59.733653 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.733639 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" path="/var/lib/kubelet/pods/97a27bca-4f25-4da5-ad54-e4dab81fe99b/volumes" Oct 30 20:07:59.734143 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.734124 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="981c36db-e723-499d-802f-cbfb370addee" path="/var/lib/kubelet/pods/981c36db-e723-499d-802f-cbfb370addee/volumes" Oct 30 20:07:59.734833 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:07:59.734818 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" path="/var/lib/kubelet/pods/9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca/volumes" Oct 30 20:08:00.055254 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:00.055164 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:01.523989 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.523964 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:08:01.564108 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564082 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-socket\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564253 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564121 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-podinfo\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564253 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564162 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-credential-socket\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564357 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564337 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-data\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564400 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564381 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-socket" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:01.564498 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564403 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-kube-api-access-mcdws\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-kube-api-access-mcdws\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564498 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564434 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87bef8f5-1dad-404f-9386-aed32dc162a8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87bef8f5-1dad-404f-9386-aed32dc162a8-istiod-ca-cert\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564498 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564465 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-credential-socket" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:01.564649 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564525 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-certs\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564649 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564543 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-data" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:01.564649 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564576 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-envoy\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564649 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564594 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-token\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-token\") pod \"87bef8f5-1dad-404f-9386-aed32dc162a8\" (UID: \"87bef8f5-1dad-404f-9386-aed32dc162a8\") " Oct 30 20:08:01.564851 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564786 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-certs" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:01.564851 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.564832 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87bef8f5-1dad-404f-9386-aed32dc162a8-istiod-ca-cert" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:01.565337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.565311 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.565337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.565337 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.565523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.565349 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.565523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.565357 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87bef8f5-1dad-404f-9386-aed32dc162a8-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.565523 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.565364 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.566631 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.566609 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-podinfo" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:01.566752 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.566714 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-envoy" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:01.567273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.567251 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-token" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:01.567341 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.567277 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-kube-api-access-mcdws" pod "87bef8f5-1dad-404f-9386-aed32dc162a8" (UID: "87bef8f5-1dad-404f-9386-aed32dc162a8"). InnerVolumeSpecName "kube-api-access-mcdws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:01.666516 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.666489 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.666621 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.666520 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.666621 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.666529 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87bef8f5-1dad-404f-9386-aed32dc162a8-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.666621 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.666541 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcdws\" (UniqueName: \"kubernetes.io/projected/87bef8f5-1dad-404f-9386-aed32dc162a8-kube-api-access-mcdws\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:01.932159 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:01.932129 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:02.006729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.006701 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:08:02.011835 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.011800 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:08:02.074932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.074683 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-kube-api-access-bdj4w\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-kube-api-access-bdj4w\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.074932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.074737 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28159db6-1d64-4199-8833-06c5eba271e1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28159db6-1d64-4199-8833-06c5eba271e1-istio-podinfo\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.074932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.074767 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-socket\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.074932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.074846 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-podinfo\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075374 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-certs\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075524 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-data\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075558 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-credential-socket\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075593 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/130f8cd2-8f57-4058-8f1f-295c0718de86-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/130f8cd2-8f57-4058-8f1f-295c0718de86-istiod-ca-cert\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075640 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-envoy\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075677 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-envoy\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075719 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-credential-socket\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075777 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-istio-token\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-istio-token\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075807 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-certs\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075865 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-kube-api-access-4bmb5\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-kube-api-access-4bmb5\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075967 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28159db6-1d64-4199-8833-06c5eba271e1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28159db6-1d64-4199-8833-06c5eba271e1-istiod-ca-cert\") pod \"28159db6-1d64-4199-8833-06c5eba271e1\" (UID: \"28159db6-1d64-4199-8833-06c5eba271e1\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.075997 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-socket\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.076041 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-data\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.077232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.076070 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-token\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-token\") pod \"130f8cd2-8f57-4058-8f1f-295c0718de86\" (UID: \"130f8cd2-8f57-4058-8f1f-295c0718de86\") " Oct 30 20:08:02.081199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.079559 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-certs" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.081199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.079856 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-credential-socket" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.081199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.080160 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-data" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.081199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.080352 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-certs" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.081199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.080525 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-socket" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.081199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.080695 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-socket" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.081199 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.080993 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28159db6-1d64-4199-8833-06c5eba271e1-istiod-ca-cert" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:02.083128 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.081663 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-data" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.083128 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.081769 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-token" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.083128 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.082023 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-credential-socket" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.083128 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.082991 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/130f8cd2-8f57-4058-8f1f-295c0718de86-istiod-ca-cert" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:02.083128 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.083087 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-kube-api-access-4bmb5" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "kube-api-access-4bmb5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.083530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.083491 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-kube-api-access-bdj4w" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "kube-api-access-bdj4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.084323 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.084206 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-envoy" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.086049 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.085812 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-envoy" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.086049 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.085889 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-podinfo" pod "130f8cd2-8f57-4058-8f1f-295c0718de86" (UID: "130f8cd2-8f57-4058-8f1f-295c0718de86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:02.087264 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.087038 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-istio-token" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.087264 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.087124 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28159db6-1d64-4199-8833-06c5eba271e1-istio-podinfo" pod "28159db6-1d64-4199-8833-06c5eba271e1" (UID: "28159db6-1d64-4199-8833-06c5eba271e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:02.181030 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.180996 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181030 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181033 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181047 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181059 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181073 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/130f8cd2-8f57-4058-8f1f-295c0718de86-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181085 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181097 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181109 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181123 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181137 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181151 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bmb5\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-kube-api-access-4bmb5\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181184 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28159db6-1d64-4199-8833-06c5eba271e1-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181198 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181212 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181225 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/130f8cd2-8f57-4058-8f1f-295c0718de86-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181238 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdj4w\" (UniqueName: \"kubernetes.io/projected/28159db6-1d64-4199-8833-06c5eba271e1-kube-api-access-bdj4w\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181253 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28159db6-1d64-4199-8833-06c5eba271e1-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.181992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.181267 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28159db6-1d64-4199-8833-06c5eba271e1-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.213159 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.212775 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:08:02.282929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282508 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-token\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-token\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.282929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282599 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-certs\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.282929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282640 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istiod-ca-cert\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.282929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282670 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-envoy\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.282929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282709 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-data\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.282929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282755 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-kube-api-access-gcvjh\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-kube-api-access-gcvjh\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.282929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282850 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-credential-socket\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.283409 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282963 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-podinfo\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.283409 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.282988 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-socket\") pod \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\" (UID: \"a0bb3aef-a539-47c9-9aa5-b989236c6a7e\") " Oct 30 20:08:02.283832 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.283797 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-socket" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.284034 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.284020 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-certs" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.284166 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.284132 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istiod-ca-cert" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:02.284300 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.284286 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-credential-socket" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.284969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.284946 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-data" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.287773 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.287740 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-podinfo" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:02.289501 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.289479 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-token" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.290721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.290428 6411 generic.go:356] "Generic (PLEG): container finished" podID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerID="e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f" exitCode=0 Oct 30 20:08:02.290721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.290479 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" Oct 30 20:08:02.290721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.290521 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" event={"ID":"a0bb3aef-a539-47c9-9aa5-b989236c6a7e","Type":"ContainerDied","Data":"e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f"} Oct 30 20:08:02.290721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.290564 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5" event={"ID":"a0bb3aef-a539-47c9-9aa5-b989236c6a7e","Type":"ContainerDied","Data":"94d5acdf99117afcb719b84028537ce347b2eb96888d2cdbbc66b06fecb4b422"} Oct 30 20:08:02.290721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.290596 6411 scope.go:122] "RemoveContainer" containerID="e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f" Oct 30 20:08:02.291552 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.291531 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-envoy" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.291649 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.291604 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-kube-api-access-gcvjh" pod "a0bb3aef-a539-47c9-9aa5-b989236c6a7e" (UID: "a0bb3aef-a539-47c9-9aa5-b989236c6a7e"). InnerVolumeSpecName "kube-api-access-gcvjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.293801 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.293575 6411 generic.go:356] "Generic (PLEG): container finished" podID="9ed090c9-2774-401e-8965-61385c1e46b0" containerID="9f126c64cd60a90eecc036674719338cea7dc22b6ddf5253a04f056863d558db" exitCode=0 Oct 30 20:08:02.293801 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.293701 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" event={"ID":"9ed090c9-2774-401e-8965-61385c1e46b0","Type":"ContainerDied","Data":"9f126c64cd60a90eecc036674719338cea7dc22b6ddf5253a04f056863d558db"} Oct 30 20:08:02.295827 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.295721 6411 generic.go:356] "Generic (PLEG): container finished" podID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerID="f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132" exitCode=0 Oct 30 20:08:02.295933 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.295860 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" event={"ID":"130f8cd2-8f57-4058-8f1f-295c0718de86","Type":"ContainerDied","Data":"f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132"} Oct 30 20:08:02.295993 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.295940 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" event={"ID":"130f8cd2-8f57-4058-8f1f-295c0718de86","Type":"ContainerDied","Data":"30fa9e110314dcf1d7af6001787d5f6fd1a61a3976d69dc78488360fcd459250"} Oct 30 20:08:02.296055 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.296013 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g" Oct 30 20:08:02.304557 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.301317 6411 generic.go:356] "Generic (PLEG): container finished" podID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerID="31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3" exitCode=0 Oct 30 20:08:02.304557 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.301427 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" event={"ID":"87bef8f5-1dad-404f-9386-aed32dc162a8","Type":"ContainerDied","Data":"31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3"} Oct 30 20:08:02.304557 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.301471 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" event={"ID":"87bef8f5-1dad-404f-9386-aed32dc162a8","Type":"ContainerDied","Data":"4c0a70b5d56d1da0028bec9946bada23204b3953253d4ffc3c4b38042d2b465f"} Oct 30 20:08:02.304557 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.301565 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd" Oct 30 20:08:02.307360 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.307296 6411 generic.go:356] "Generic (PLEG): container finished" podID="28159db6-1d64-4199-8833-06c5eba271e1" containerID="65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350" exitCode=0 Oct 30 20:08:02.307947 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.307884 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" event={"ID":"28159db6-1d64-4199-8833-06c5eba271e1","Type":"ContainerDied","Data":"65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350"} Oct 30 20:08:02.308039 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.307969 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" event={"ID":"28159db6-1d64-4199-8833-06c5eba271e1","Type":"ContainerDied","Data":"e8f5c8088dfb12b5da5c4525e7f00f8743ee536023ba199659c9a0c9bf3f7784"} Oct 30 20:08:02.308105 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.308054 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x" Oct 30 20:08:02.312829 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.310919 6411 scope.go:122] "RemoveContainer" containerID="10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116" Oct 30 20:08:02.312829 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.312577 6411 generic.go:356] "Generic (PLEG): container finished" podID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerID="d1d48d91f9feac5515189b72f794deed33786fb80e12767afdc701f4fa62916d" exitCode=0 Oct 30 20:08:02.312829 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.312604 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" event={"ID":"4c4f744d-54e4-47d1-944b-419a9d845c8b","Type":"ContainerDied","Data":"d1d48d91f9feac5515189b72f794deed33786fb80e12767afdc701f4fa62916d"} Oct 30 20:08:02.322960 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.322505 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:08:02.328873 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.328853 6411 scope.go:122] "RemoveContainer" containerID="e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f" Oct 30 20:08:02.329521 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:02.329490 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f\": container with ID starting with e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f not found: ID does not exist" containerID="e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f" Oct 30 20:08:02.329620 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.329534 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f"} err="failed to get container status \"e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f\": rpc error: code = NotFound desc = could not find container \"e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f\": container with ID starting with e3e4b10ca9b82066b6ffef95e6ae44597858e92a58e0ca8eba3227b70487036f not found: ID does not exist" Oct 30 20:08:02.329620 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.329561 6411 scope.go:122] "RemoveContainer" containerID="10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116" Oct 30 20:08:02.329870 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:02.329845 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116\": container with ID starting with 10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116 not found: ID does not exist" containerID="10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116" Oct 30 20:08:02.329988 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.329879 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116"} err="failed to get container status \"10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116\": rpc error: code = NotFound desc = could not find container \"10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116\": container with ID starting with 10ea6585cc8048c810428ff8edaf233e0f1f1d7c4b666b03eb7d77304a132116 not found: ID does not exist" Oct 30 20:08:02.329988 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.329938 6411 scope.go:122] "RemoveContainer" containerID="f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132" Oct 30 20:08:02.341885 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.341860 6411 scope.go:122] "RemoveContainer" containerID="f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132" Oct 30 20:08:02.342276 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:02.342248 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132\": container with ID starting with f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132 not found: ID does not exist" containerID="f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132" Oct 30 20:08:02.342367 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.342283 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132"} err="failed to get container status \"f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132\": rpc error: code = NotFound desc = could not find container \"f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132\": container with ID starting with f855ce63726772269b02305a40a756a3c7be20b5ce91c6bd0ab9c8d2e0b54132 not found: ID does not exist" Oct 30 20:08:02.342367 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.342309 6411 scope.go:122] "RemoveContainer" containerID="31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3" Oct 30 20:08:02.379927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.379884 6411 scope.go:122] "RemoveContainer" containerID="31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3" Oct 30 20:08:02.380326 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:02.380303 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3\": container with ID starting with 31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3 not found: ID does not exist" containerID="31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3" Oct 30 20:08:02.380427 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.380336 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3"} err="failed to get container status \"31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3\": rpc error: code = NotFound desc = could not find container \"31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3\": container with ID starting with 31b7f2d8d9a62372f6f10c30c674d80de040356b7dbfba7613a098e4661cc9a3 not found: ID does not exist" Oct 30 20:08:02.380427 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.380360 6411 scope.go:122] "RemoveContainer" containerID="65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350" Oct 30 20:08:02.384800 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.384772 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-kube-api-access-ztz86\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-kube-api-access-ztz86\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.384990 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.384968 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-socket\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.385061 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.385041 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-istio-token\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-istio-token\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.385121 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.385073 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-certs\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.385121 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.385115 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-data\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.385228 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.385186 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9ed090c9-2774-401e-8965-61385c1e46b0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed090c9-2774-401e-8965-61385c1e46b0-istio-podinfo\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.385228 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.385221 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-credential-socket\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.385330 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.385290 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-envoy\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.385330 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.385318 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ed090c9-2774-401e-8965-61385c1e46b0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed090c9-2774-401e-8965-61385c1e46b0-istiod-ca-cert\") pod \"9ed090c9-2774-401e-8965-61385c1e46b0\" (UID: \"9ed090c9-2774-401e-8965-61385c1e46b0\") " Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.385712 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-data" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386045 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ed090c9-2774-401e-8965-61385c1e46b0-istiod-ca-cert" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386154 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-credential-socket" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386432 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-certs" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386459 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-socket" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386592 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386611 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386626 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386639 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcvjh\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-kube-api-access-gcvjh\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386652 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386666 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386679 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386692 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386705 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.386768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.386733 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0bb3aef-a539-47c9-9aa5-b989236c6a7e-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.390188 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.390116 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd"] Oct 30 20:08:02.390593 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.390568 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-kube-api-access-ztz86" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "kube-api-access-ztz86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.390829 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.390799 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ed090c9-2774-401e-8965-61385c1e46b0-istio-podinfo" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:02.391160 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.391137 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-envoy" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.393419 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.393392 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-istio-token" pod "9ed090c9-2774-401e-8965-61385c1e46b0" (UID: "9ed090c9-2774-401e-8965-61385c1e46b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.399016 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.398978 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-fs8qd"] Oct 30 20:08:02.402520 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.402475 6411 scope.go:122] "RemoveContainer" containerID="65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350" Oct 30 20:08:02.403057 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:02.402958 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350\": container with ID starting with 65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350 not found: ID does not exist" containerID="65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350" Oct 30 20:08:02.403057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.402994 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350"} err="failed to get container status \"65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350\": rpc error: code = NotFound desc = could not find container \"65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350\": container with ID starting with 65092bafd1c5896f829ccc81d428617e698873bf74aefc41fd59117714953350 not found: ID does not exist" Oct 30 20:08:02.434918 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.434810 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g"] Oct 30 20:08:02.444116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.444069 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-hpt7g"] Oct 30 20:08:02.490425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.488445 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.490425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.489169 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed090c9-2774-401e-8965-61385c1e46b0-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.490425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.489199 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztz86\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-kube-api-access-ztz86\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.490425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.489216 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.490425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.489232 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed090c9-2774-401e-8965-61385c1e46b0-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.490425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.489244 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.490425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.489256 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed090c9-2774-401e-8965-61385c1e46b0-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.490425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.489268 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed090c9-2774-401e-8965-61385c1e46b0-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.505969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.493986 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x"] Oct 30 20:08:02.505969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.503406 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-hzk8x"] Oct 30 20:08:02.557148 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.557103 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:08:02.658880 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.658852 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5"] Oct 30 20:08:02.665348 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.665323 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f1f016-openshift-default-6c85499cc-78vd5"] Oct 30 20:08:02.691247 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691175 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-certs\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.691247 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691210 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-podinfo\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.691247 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691238 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-socket\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.691502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691269 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-token\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-token\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.691502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691290 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c4f744d-54e4-47d1-944b-419a9d845c8b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4f744d-54e4-47d1-944b-419a9d845c8b-istiod-ca-cert\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.691502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691337 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-data\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.691502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691468 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-certs" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.691694 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691540 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-kube-api-access-vv9gl\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-kube-api-access-vv9gl\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.691694 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691626 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-data" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.691694 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691629 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-socket" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.691813 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691745 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c4f744d-54e4-47d1-944b-419a9d845c8b-istiod-ca-cert" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:02.691813 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691760 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-envoy\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.691944 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.691825 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-credential-socket\") pod \"4c4f744d-54e4-47d1-944b-419a9d845c8b\" (UID: \"4c4f744d-54e4-47d1-944b-419a9d845c8b\") " Oct 30 20:08:02.692169 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.692132 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-credential-socket" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.692623 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.692598 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.692730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.692629 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.692730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.692645 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.692730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.692659 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4f744d-54e4-47d1-944b-419a9d845c8b-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.692730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.692673 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.693941 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.693882 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-token" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.694029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.693951 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-kube-api-access-vv9gl" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "kube-api-access-vv9gl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:02.694106 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.694083 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-podinfo" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:02.694144 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.694084 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-envoy" pod "4c4f744d-54e4-47d1-944b-419a9d845c8b" (UID: "4c4f744d-54e4-47d1-944b-419a9d845c8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:02.793684 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.793646 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.793684 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.793677 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.793684 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.793689 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:02.793891 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:02.793702 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv9gl\" (UniqueName: \"kubernetes.io/projected/4c4f744d-54e4-47d1-944b-419a9d845c8b-kube-api-access-vv9gl\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:03.319777 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.319753 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" Oct 30 20:08:03.320011 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.319746 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx" event={"ID":"4c4f744d-54e4-47d1-944b-419a9d845c8b","Type":"ContainerDied","Data":"f1323a987efb1771b2fba5f0c775a38dcef0254399addc4408980befcee8f850"} Oct 30 20:08:03.320011 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.319889 6411 scope.go:122] "RemoveContainer" containerID="d1d48d91f9feac5515189b72f794deed33786fb80e12767afdc701f4fa62916d" Oct 30 20:08:03.322716 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.322692 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" Oct 30 20:08:03.322857 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.322709 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn" event={"ID":"9ed090c9-2774-401e-8965-61385c1e46b0","Type":"ContainerDied","Data":"a8e465e4d636d1ce5c96a25b03752e9b5ea5bf0130e231a6e86b39015b867b95"} Oct 30 20:08:03.333516 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.333491 6411 scope.go:122] "RemoveContainer" containerID="9f126c64cd60a90eecc036674719338cea7dc22b6ddf5253a04f056863d558db" Oct 30 20:08:03.391955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.391920 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx"] Oct 30 20:08:03.404044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.404015 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-vdsqx"] Oct 30 20:08:03.437680 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.437650 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn"] Oct 30 20:08:03.443528 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.443501 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-626870-openshift-default-7cd649d59c-x4wxn"] Oct 30 20:08:03.734349 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.734320 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" path="/var/lib/kubelet/pods/130f8cd2-8f57-4058-8f1f-295c0718de86/volumes" Oct 30 20:08:03.734854 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.734839 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28159db6-1d64-4199-8833-06c5eba271e1" path="/var/lib/kubelet/pods/28159db6-1d64-4199-8833-06c5eba271e1/volumes" Oct 30 20:08:03.735276 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.735263 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" path="/var/lib/kubelet/pods/4c4f744d-54e4-47d1-944b-419a9d845c8b/volumes" Oct 30 20:08:03.736032 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.736013 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" path="/var/lib/kubelet/pods/87bef8f5-1dad-404f-9386-aed32dc162a8/volumes" Oct 30 20:08:03.736434 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.736418 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" path="/var/lib/kubelet/pods/9ed090c9-2774-401e-8965-61385c1e46b0/volumes" Oct 30 20:08:03.736830 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.736816 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" path="/var/lib/kubelet/pods/a0bb3aef-a539-47c9-9aa5-b989236c6a7e/volumes" Oct 30 20:08:03.755959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.755929 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 20:08:03.756076 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.755989 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 20:08:03.917745 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.917710 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:03.962383 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.962335 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 20:08:03.962526 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:03.962411 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 20:08:04.151849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.151806 6411 patch_prober.go:29] interesting pod/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Oct 30 20:08:04.152034 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.151869 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 20:08:04.643776 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.643748 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:08:04.710996 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.710967 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:08:04.718462 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.718439 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:08:04.729017 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.728991 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-token\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-token\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.729145 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.729035 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istiod-ca-cert\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.729145 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.729068 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-podinfo\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.729145 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.729100 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-credential-socket\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.729308 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.729297 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-kube-api-access-468lt\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-kube-api-access-468lt\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.729370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.729330 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-data\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.729440 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.729417 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-socket\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.729762 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.729533 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-certs\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.729762 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.729592 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-envoy\") pod \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\" (UID: \"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c\") " Oct 30 20:08:04.730091 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.730066 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-socket" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.730222 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.730199 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.730295 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.730262 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-data" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.730403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.730375 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-certs" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.730557 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.730530 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istiod-ca-cert" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:04.730679 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.730658 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-credential-socket" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.733369 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.733346 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-envoy" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.733577 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.733552 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-token" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:04.733849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.733815 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-kube-api-access-468lt" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "kube-api-access-468lt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:04.734290 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.734266 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-podinfo" pod "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" (UID: "6aac11e2-19d4-490f-a67f-af5e5ad9ba1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:04.831101 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831008 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istiod-ca-cert\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.831101 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831077 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-socket\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831118 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-certs\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831162 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-kube-api-access-s7k5x\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-kube-api-access-s7k5x\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831251 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-podinfo\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831288 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-podinfo\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831340 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-envoy\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831378 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-data\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831451 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istiod-ca-cert" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831482 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-token\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-token\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831520 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-certs\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831567 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-data\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.831614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831600 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-socket\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.832175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831649 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8904408-b320-477b-9ab8-42498ca0a1c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8904408-b320-477b-9ab8-42498ca0a1c5-istiod-ca-cert\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.832175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831676 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-credential-socket\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.832175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831715 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-token\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-token\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.832175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831751 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-envoy\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.832175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831793 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-credential-socket\") pod \"b8904408-b320-477b-9ab8-42498ca0a1c5\" (UID: \"b8904408-b320-477b-9ab8-42498ca0a1c5\") " Oct 30 20:08:04.832175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831882 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-kube-api-access-mc6sk\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-kube-api-access-mc6sk\") pod \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\" (UID: \"aef66d88-ee2e-4c41-aeaa-bf46db26bc89\") " Oct 30 20:08:04.832175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.831944 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-certs" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.832573 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832545 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-468lt\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-kube-api-access-468lt\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832579 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832594 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832608 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832621 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832633 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832645 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832985 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832657 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832985 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832654 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-data" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.832985 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832669 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.832985 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832683 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.833176 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.832984 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-certs" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.833176 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.833119 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-data" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.834261 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.833470 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-socket" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.834261 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.833616 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8904408-b320-477b-9ab8-42498ca0a1c5-istiod-ca-cert" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:04.834261 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.833859 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-socket" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.834261 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.834130 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-credential-socket" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.840384 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.834438 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-credential-socket" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.840384 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.837784 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-envoy" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.842619 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.842574 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-envoy" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:04.846403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.846298 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-kube-api-access-s7k5x" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "kube-api-access-s7k5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:04.846403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.846339 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-token" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:04.846403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.846360 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-token" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:04.846403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.846367 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-podinfo" pod "b8904408-b320-477b-9ab8-42498ca0a1c5" (UID: "b8904408-b320-477b-9ab8-42498ca0a1c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:04.846854 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.846475 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-podinfo" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:04.848021 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.847996 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-kube-api-access-mc6sk" pod "aef66d88-ee2e-4c41-aeaa-bf46db26bc89" (UID: "aef66d88-ee2e-4c41-aeaa-bf46db26bc89"). InnerVolumeSpecName "kube-api-access-mc6sk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:04.934204 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934170 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934204 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934204 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934217 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934229 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934239 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934249 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934260 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934272 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8904408-b320-477b-9ab8-42498ca0a1c5-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934286 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934298 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934309 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934320 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934331 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc6sk\" (UniqueName: \"kubernetes.io/projected/aef66d88-ee2e-4c41-aeaa-bf46db26bc89-kube-api-access-mc6sk\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934343 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934354 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8904408-b320-477b-9ab8-42498ca0a1c5-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:04.934456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:04.934368 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7k5x\" (UniqueName: \"kubernetes.io/projected/b8904408-b320-477b-9ab8-42498ca0a1c5-kube-api-access-s7k5x\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:05.100065 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:05.099968 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:08:05.343852 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.343813 6411 generic.go:356] "Generic (PLEG): container finished" podID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerID="7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9" exitCode=0 Oct 30 20:08:05.344057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.343877 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" Oct 30 20:08:05.344057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.343914 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" event={"ID":"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c","Type":"ContainerDied","Data":"7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9"} Oct 30 20:08:05.344057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.343976 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd" event={"ID":"6aac11e2-19d4-490f-a67f-af5e5ad9ba1c","Type":"ContainerDied","Data":"08b10df81267ace365159745798294f4412f1a51888c5d4cdccdcd822a6e1784"} Oct 30 20:08:05.344057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.343998 6411 scope.go:122] "RemoveContainer" containerID="7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9" Oct 30 20:08:05.351290 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.351230 6411 generic.go:356] "Generic (PLEG): container finished" podID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerID="d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f" exitCode=0 Oct 30 20:08:05.351290 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.351267 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" event={"ID":"aef66d88-ee2e-4c41-aeaa-bf46db26bc89","Type":"ContainerDied","Data":"d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f"} Oct 30 20:08:05.351451 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.351294 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" Oct 30 20:08:05.351451 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.351330 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv" event={"ID":"aef66d88-ee2e-4c41-aeaa-bf46db26bc89","Type":"ContainerDied","Data":"4a7734b94ccb6c8d6a5a8c9674c261084066b8e9fd8d4ad8319a178ff47d2a24"} Oct 30 20:08:05.353076 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.353047 6411 generic.go:356] "Generic (PLEG): container finished" podID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerID="c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f" exitCode=0 Oct 30 20:08:05.353186 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.353097 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" Oct 30 20:08:05.353263 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.353097 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" event={"ID":"b8904408-b320-477b-9ab8-42498ca0a1c5","Type":"ContainerDied","Data":"c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f"} Oct 30 20:08:05.353263 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.353227 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf" event={"ID":"b8904408-b320-477b-9ab8-42498ca0a1c5","Type":"ContainerDied","Data":"7da4604130c0a1cdd2a2a17c102451b4039b602baa22791706b1140efd1c2962"} Oct 30 20:08:05.354858 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.354796 6411 scope.go:122] "RemoveContainer" containerID="7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9" Oct 30 20:08:05.362275 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:05.362210 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9\": container with ID starting with 7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9 not found: ID does not exist" containerID="7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9" Oct 30 20:08:05.362275 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.362249 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9"} err="failed to get container status \"7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9\": rpc error: code = NotFound desc = could not find container \"7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9\": container with ID starting with 7af64051228f881818ab58e5b217fb9edfd0c73e29aa93b8c79aa705e93d37b9 not found: ID does not exist" Oct 30 20:08:05.362275 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.362271 6411 scope.go:122] "RemoveContainer" containerID="d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f" Oct 30 20:08:05.383377 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.383360 6411 scope.go:122] "RemoveContainer" containerID="d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f" Oct 30 20:08:05.383716 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:05.383695 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f\": container with ID starting with d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f not found: ID does not exist" containerID="d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f" Oct 30 20:08:05.383803 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.383726 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f"} err="failed to get container status \"d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f\": rpc error: code = NotFound desc = could not find container \"d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f\": container with ID starting with d49cc06f212c68c426adc787de5d518b5110e8a9c9539a5210877ce2f324969f not found: ID does not exist" Oct 30 20:08:05.383803 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.383748 6411 scope.go:122] "RemoveContainer" containerID="c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f" Oct 30 20:08:05.393657 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.393637 6411 scope.go:122] "RemoveContainer" containerID="c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f" Oct 30 20:08:05.393972 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:05.393949 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f\": container with ID starting with c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f not found: ID does not exist" containerID="c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f" Oct 30 20:08:05.394047 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.393978 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f"} err="failed to get container status \"c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f\": rpc error: code = NotFound desc = could not find container \"c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f\": container with ID starting with c8478ea5aa71d3b88a8419664ddeb14241822abf53b5a03e762359fbf1d5209f not found: ID does not exist" Oct 30 20:08:05.405044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.405019 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd"] Oct 30 20:08:05.427967 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.427937 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zqtgd"] Oct 30 20:08:05.485663 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.485631 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv"] Oct 30 20:08:05.490377 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.490344 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-6z7vv"] Oct 30 20:08:05.527936 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.527889 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf"] Oct 30 20:08:05.537459 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.536388 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-912f24-openshift-default-5f56ff5f6c-zn2kf"] Oct 30 20:08:05.736922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.735988 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:05.740002 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.739970 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" path="/var/lib/kubelet/pods/6aac11e2-19d4-490f-a67f-af5e5ad9ba1c/volumes" Oct 30 20:08:05.740698 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.740674 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" path="/var/lib/kubelet/pods/aef66d88-ee2e-4c41-aeaa-bf46db26bc89/volumes" Oct 30 20:08:05.741325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.741303 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" path="/var/lib/kubelet/pods/b8904408-b320-477b-9ab8-42498ca0a1c5/volumes" Oct 30 20:08:05.919937 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:05.919866 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:07.547792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:07.547756 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:07.724735 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:07.724695 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" podUID="aeddd424-8ad6-40dd-aaec-68195c43415e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 20:08:09.362957 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:09.362892 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:10.890785 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:10.890753 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5"] Oct 30 20:08:10.891270 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:10.891020 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" podUID="fc4dca37-ec59-4707-bd27-0eb7d6b556af" containerName="maas-api" containerID="cri-o://36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288" gracePeriod=30 Oct 30 20:08:10.978356 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:10.978326 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:11.173326 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.173300 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:08:11.236334 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.235215 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-metrics-certs\") pod \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " Oct 30 20:08:11.236334 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.235263 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-maas-api-tls\") pod \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " Oct 30 20:08:11.236334 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.235434 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc4dca37-ec59-4707-bd27-0eb7d6b556af-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4dca37-ec59-4707-bd27-0eb7d6b556af-openshift-service-ca\") pod \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " Oct 30 20:08:11.236334 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.235515 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc4dca37-ec59-4707-bd27-0eb7d6b556af-kube-api-access-4vnd6\" (UniqueName: \"kubernetes.io/projected/fc4dca37-ec59-4707-bd27-0eb7d6b556af-kube-api-access-4vnd6\") pod \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\" (UID: \"fc4dca37-ec59-4707-bd27-0eb7d6b556af\") " Oct 30 20:08:11.237403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.237358 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4dca37-ec59-4707-bd27-0eb7d6b556af-openshift-service-ca" pod "fc4dca37-ec59-4707-bd27-0eb7d6b556af" (UID: "fc4dca37-ec59-4707-bd27-0eb7d6b556af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:11.241875 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.241831 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-metrics-certs" pod "fc4dca37-ec59-4707-bd27-0eb7d6b556af" (UID: "fc4dca37-ec59-4707-bd27-0eb7d6b556af"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:08:11.277817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.277779 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-maas-api-tls" pod "fc4dca37-ec59-4707-bd27-0eb7d6b556af" (UID: "fc4dca37-ec59-4707-bd27-0eb7d6b556af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:08:11.288133 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.288089 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc4dca37-ec59-4707-bd27-0eb7d6b556af-kube-api-access-4vnd6" pod "fc4dca37-ec59-4707-bd27-0eb7d6b556af" (UID: "fc4dca37-ec59-4707-bd27-0eb7d6b556af"). InnerVolumeSpecName "kube-api-access-4vnd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:11.337828 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.337794 6411 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4dca37-ec59-4707-bd27-0eb7d6b556af-openshift-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:11.338271 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.338250 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vnd6\" (UniqueName: \"kubernetes.io/projected/fc4dca37-ec59-4707-bd27-0eb7d6b556af-kube-api-access-4vnd6\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:11.338564 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.338374 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:11.338694 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.338679 6411 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4dca37-ec59-4707-bd27-0eb7d6b556af-maas-api-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:11.394223 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.394127 6411 generic.go:356] "Generic (PLEG): container finished" podID="fc4dca37-ec59-4707-bd27-0eb7d6b556af" containerID="36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288" exitCode=0 Oct 30 20:08:11.394223 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.394163 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" event={"ID":"fc4dca37-ec59-4707-bd27-0eb7d6b556af","Type":"ContainerDied","Data":"36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288"} Oct 30 20:08:11.394223 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.394189 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" Oct 30 20:08:11.394223 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.394217 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5" event={"ID":"fc4dca37-ec59-4707-bd27-0eb7d6b556af","Type":"ContainerDied","Data":"98380ed1d254e0326b3242695e144b268c4777568461b0d30d243cc803cd2168"} Oct 30 20:08:11.394557 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.394240 6411 scope.go:122] "RemoveContainer" containerID="36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288" Oct 30 20:08:11.405644 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.405624 6411 scope.go:122] "RemoveContainer" containerID="36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288" Oct 30 20:08:11.405946 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:11.405921 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288\": container with ID starting with 36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288 not found: ID does not exist" containerID="36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288" Oct 30 20:08:11.405994 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.405958 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288"} err="failed to get container status \"36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288\": rpc error: code = NotFound desc = could not find container \"36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288\": container with ID starting with 36678f32a0c11ecbb69380e1eb9af78dfcfd66bdaff9882659cf27396c791288 not found: ID does not exist" Oct 30 20:08:11.430329 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.430305 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:11.478184 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.478151 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5"] Oct 30 20:08:11.485470 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.485447 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-62f4a1-54dd9b7796-nnql5"] Oct 30 20:08:11.658578 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.658268 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr"] Oct 30 20:08:11.659155 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659077 6411 cpu_manager.go:414] "containerMap: removing container" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" Oct 30 20:08:11.659155 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659101 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" Oct 30 20:08:11.659155 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659120 6411 cpu_manager.go:414] "containerMap: removing container" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" Oct 30 20:08:11.659155 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659128 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" Oct 30 20:08:11.659798 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659766 6411 cpu_manager.go:414] "containerMap: removing container" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" Oct 30 20:08:11.659798 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659789 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659809 6411 cpu_manager.go:414] "containerMap: removing container" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659817 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659828 6411 cpu_manager.go:414] "containerMap: removing container" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659835 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659861 6411 cpu_manager.go:414] "containerMap: removing container" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659867 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659878 6411 cpu_manager.go:414] "containerMap: removing container" podUID="b39478e6-123e-48ad-b7f4-d48da825d8ee" containerName="maas-api" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659887 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b39478e6-123e-48ad-b7f4-d48da825d8ee" containerName="maas-api" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659914 6411 cpu_manager.go:414] "containerMap: removing container" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659923 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659935 6411 cpu_manager.go:414] "containerMap: removing container" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659942 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659956 6411 cpu_manager.go:414] "containerMap: removing container" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659963 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" Oct 30 20:08:11.659971 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659975 6411 cpu_manager.go:414] "containerMap: removing container" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.659982 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660002 6411 cpu_manager.go:414] "containerMap: removing container" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660009 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660019 6411 cpu_manager.go:414] "containerMap: removing container" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660026 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660041 6411 cpu_manager.go:414] "containerMap: removing container" podUID="5c095837-e094-43c1-aa14-5715094b2bbb" containerName="payload-processing" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660048 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c095837-e094-43c1-aa14-5715094b2bbb" containerName="payload-processing" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660058 6411 cpu_manager.go:414] "containerMap: removing container" podUID="fda073ed-6c46-42ef-a592-c99c980f8885" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660065 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fda073ed-6c46-42ef-a592-c99c980f8885" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660075 6411 cpu_manager.go:414] "containerMap: removing container" podUID="fc4dca37-ec59-4707-bd27-0eb7d6b556af" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660082 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc4dca37-ec59-4707-bd27-0eb7d6b556af" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660091 6411 cpu_manager.go:414] "containerMap: removing container" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660097 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660108 6411 cpu_manager.go:414] "containerMap: removing container" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660114 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660124 6411 cpu_manager.go:414] "containerMap: removing container" podUID="4a92199c-6d6c-4020-b877-8e585a87cd28" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660131 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a92199c-6d6c-4020-b877-8e585a87cd28" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660139 6411 cpu_manager.go:414] "containerMap: removing container" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660149 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660162 6411 cpu_manager.go:414] "containerMap: removing container" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660169 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660286 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="fda073ed-6c46-42ef-a592-c99c980f8885" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660298 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc4dca37-ec59-4707-bd27-0eb7d6b556af" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660315 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="130f8cd2-8f57-4058-8f1f-295c0718de86" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660323 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ed090c9-2774-401e-8965-61385c1e46b0" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660331 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a92199c-6d6c-4020-b877-8e585a87cd28" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660339 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="28159db6-1d64-4199-8833-06c5eba271e1" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660348 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="6aac11e2-19d4-490f-a67f-af5e5ad9ba1c" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660358 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="93138c42-74ff-4d13-b362-4f859beb67df" containerName="main" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660366 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="00f5fddd-2204-412a-bee6-ddd2dbe13213" containerName="main" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660374 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="87bef8f5-1dad-404f-9386-aed32dc162a8" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660381 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="981c36db-e723-499d-802f-cbfb370addee" containerName="main" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660389 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660399 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0bb3aef-a539-47c9-9aa5-b989236c6a7e" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660407 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="b39478e6-123e-48ad-b7f4-d48da825d8ee" containerName="maas-api" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660415 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="97a27bca-4f25-4da5-ad54-e4dab81fe99b" containerName="main" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660423 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="aef66d88-ee2e-4c41-aeaa-bf46db26bc89" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660432 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f940223-245c-4132-880c-dd2f75dfe9cf" containerName="main" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660441 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8904408-b320-477b-9ab8-42498ca0a1c5" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660452 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f3e0a8f-33c8-4a22-a9f8-d1cf75e60fca" containerName="main" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660461 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c4f744d-54e4-47d1-944b-419a9d845c8b" containerName="istio-proxy" Oct 30 20:08:11.660756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.660470 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c095837-e094-43c1-aa14-5715094b2bbb" containerName="payload-processing" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.743748 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745140 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745195 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745233 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745287 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745321 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8vkb\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-kube-api-access-d8vkb\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745359 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745415 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745450 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.758064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.745508 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.820547 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc4dca37-ec59-4707-bd27-0eb7d6b556af" path="/var/lib/kubelet/pods/fc4dca37-ec59-4707-bd27-0eb7d6b556af/volumes" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.845521 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846309 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846420 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846481 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846519 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846583 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846619 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8vkb\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-kube-api-access-d8vkb\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846662 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846737 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846779 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.846858 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.847814 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.848388 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.851206 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.851457 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.851949 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.855781 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.877006 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.875363 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9"] Oct 30 20:08:11.887412 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.883882 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr"] Oct 30 20:08:11.887412 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.883968 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9"] Oct 30 20:08:11.887412 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.884077 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.897852 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.897778 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.899840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.899803 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8vkb\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-kube-api-access-d8vkb\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:11.947797 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.947761 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.948020 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.948001 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.948100 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.948065 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.948138 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.948116 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.948268 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.948241 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.948331 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.948318 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.948441 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.948421 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.948525 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.948503 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:11.948667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:11.948650 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7fbn\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-kube-api-access-m7fbn\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.049792 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.050036 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.050122 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.050192 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.050257 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.050316 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.050358 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.050435 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.051667 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.050506 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7fbn\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-kube-api-access-m7fbn\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.054442 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.054058 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istiod-ca-cert\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.056237 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.055692 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.056237 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.056185 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-credential-socket\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.057289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.056533 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-data\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.057289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.056601 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-certs\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.073707 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.073619 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-podinfo\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.110923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.104285 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-envoy\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.115869 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.115837 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-token\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.116665 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.116637 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7fbn\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-kube-api-access-m7fbn\") pod \"e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.175497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.175415 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:12.206469 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.206438 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:12.324447 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.324010 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:12.358676 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.358652 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:08:12.406149 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.406117 6411 generic.go:356] "Generic (PLEG): container finished" podID="aeddd424-8ad6-40dd-aaec-68195c43415e" containerID="0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569" exitCode=0 Oct 30 20:08:12.406345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.406217 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" event={"ID":"aeddd424-8ad6-40dd-aaec-68195c43415e","Type":"ContainerDied","Data":"0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569"} Oct 30 20:08:12.406345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.406225 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" Oct 30 20:08:12.406345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.406269 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk" event={"ID":"aeddd424-8ad6-40dd-aaec-68195c43415e","Type":"ContainerDied","Data":"0b8ab6561806c4e9e5ad9573bd53f2fa41deb5a7d060f770e8210a4715583719"} Oct 30 20:08:12.406345 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.406298 6411 scope.go:122] "RemoveContainer" containerID="0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569" Oct 30 20:08:12.425550 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.425420 6411 scope.go:122] "RemoveContainer" containerID="0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569" Oct 30 20:08:12.426055 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:12.425978 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569\": container with ID starting with 0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569 not found: ID does not exist" containerID="0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569" Oct 30 20:08:12.426055 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.426018 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569"} err="failed to get container status \"0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569\": rpc error: code = NotFound desc = could not find container \"0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569\": container with ID starting with 0d410edb29cad08e1a9f510a1737e2804aca17fb180f42d7523d590a650bd569 not found: ID does not exist" Oct 30 20:08:12.456723 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.456699 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aeddd424-8ad6-40dd-aaec-68195c43415e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aeddd424-8ad6-40dd-aaec-68195c43415e-plugins-config-volume\") pod \"aeddd424-8ad6-40dd-aaec-68195c43415e\" (UID: \"aeddd424-8ad6-40dd-aaec-68195c43415e\") " Oct 30 20:08:12.456841 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.456809 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aeddd424-8ad6-40dd-aaec-68195c43415e-kube-api-access-zhp94\" (UniqueName: \"kubernetes.io/projected/aeddd424-8ad6-40dd-aaec-68195c43415e-kube-api-access-zhp94\") pod \"aeddd424-8ad6-40dd-aaec-68195c43415e\" (UID: \"aeddd424-8ad6-40dd-aaec-68195c43415e\") " Oct 30 20:08:12.457124 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.457098 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeddd424-8ad6-40dd-aaec-68195c43415e-plugins-config-volume" pod "aeddd424-8ad6-40dd-aaec-68195c43415e" (UID: "aeddd424-8ad6-40dd-aaec-68195c43415e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:12.457288 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.457273 6411 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aeddd424-8ad6-40dd-aaec-68195c43415e-plugins-config-volume\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:12.459046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.459014 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeddd424-8ad6-40dd-aaec-68195c43415e-kube-api-access-zhp94" pod "aeddd424-8ad6-40dd-aaec-68195c43415e" (UID: "aeddd424-8ad6-40dd-aaec-68195c43415e"). InnerVolumeSpecName "kube-api-access-zhp94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:12.522502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.522470 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr"] Oct 30 20:08:12.524070 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:08:12.524043 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda210c8b6_cbb5_4ed7_b271_49e4912d42e6.slice/crio-afd6adbcc6adf89711a34ce5598dd341b470413d202db4a55e780dc53a104d30 WatchSource:0}: Error finding container afd6adbcc6adf89711a34ce5598dd341b470413d202db4a55e780dc53a104d30: Status 404 returned error can't find the container with id afd6adbcc6adf89711a34ce5598dd341b470413d202db4a55e780dc53a104d30 Oct 30 20:08:12.527353 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.527315 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:08:12.527475 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.527399 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:08:12.527475 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.527445 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:08:12.545546 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.545516 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:12.558912 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.558848 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhp94\" (UniqueName: \"kubernetes.io/projected/aeddd424-8ad6-40dd-aaec-68195c43415e-kube-api-access-zhp94\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:12.592230 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.579703 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9"] Oct 30 20:08:12.620497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.619048 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:08:12.620497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.619143 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:08:12.620497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.619190 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:08:12.779082 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.779042 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk"] Oct 30 20:08:12.788524 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.788494 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-62f4a1-548465b667-w6slk"] Oct 30 20:08:12.886866 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:12.886832 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:13.191209 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:13.191177 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:13.416681 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:13.416645 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" event={"ID":"a210c8b6-cbb5-4ed7-b271-49e4912d42e6","Type":"ContainerStarted","Data":"17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf"} Oct 30 20:08:13.416853 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:13.416709 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" event={"ID":"a210c8b6-cbb5-4ed7-b271-49e4912d42e6","Type":"ContainerStarted","Data":"afd6adbcc6adf89711a34ce5598dd341b470413d202db4a55e780dc53a104d30"} Oct 30 20:08:13.419576 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:13.419550 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" event={"ID":"546bae1b-d60e-47b5-8414-6fd7a0d66a5e","Type":"ContainerStarted","Data":"320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9"} Oct 30 20:08:13.419740 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:13.419597 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" event={"ID":"546bae1b-d60e-47b5-8414-6fd7a0d66a5e","Type":"ContainerStarted","Data":"7c4119d1c8975313c993cfa47eea97f3e1ccf0e2938f87b0d7b0cfc48dcb4625"} Oct 30 20:08:13.486988 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:13.484249 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podStartSLOduration=2.484230543 podStartE2EDuration="2.484230543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:08:13.477019184 +0000 UTC m=+1209.877191191" watchObservedRunningTime="2026-10-30 20:08:13.484230543 +0000 UTC m=+1209.884402551" Oct 30 20:08:13.540741 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:13.540676 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podStartSLOduration=2.540655795 podStartE2EDuration="2.540655795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:08:13.53753144 +0000 UTC m=+1209.937703456" watchObservedRunningTime="2026-10-30 20:08:13.540655795 +0000 UTC m=+1209.940827803" Oct 30 20:08:13.734974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:13.734940 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aeddd424-8ad6-40dd-aaec-68195c43415e" path="/var/lib/kubelet/pods/aeddd424-8ad6-40dd-aaec-68195c43415e/volumes" Oct 30 20:08:14.176053 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:14.176016 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:14.177274 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:14.177249 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Oct 30 20:08:14.177350 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:14.177304 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Oct 30 20:08:14.211273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:14.211244 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:14.212002 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:14.211974 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:14.212172 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:14.212150 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:14.695772 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:14.695742 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:15.176748 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:15.176714 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Oct 30 20:08:15.176943 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:15.176766 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Oct 30 20:08:15.222920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:15.206863 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:15.222920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:15.206966 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:16.176560 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:16.176525 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Oct 30 20:08:16.176718 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:16.176583 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Oct 30 20:08:16.232031 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:16.231997 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:16.232609 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:16.232576 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:16.751935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:16.742073 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:17.176189 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:17.176156 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Oct 30 20:08:17.176393 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:17.176228 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Oct 30 20:08:17.206920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:17.206781 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:17.206920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:17.206837 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:17.357535 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:17.357359 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:17.673245 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:17.639806 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:18.176744 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:18.176706 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Oct 30 20:08:18.176963 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:18.176783 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Oct 30 20:08:18.212922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:18.210604 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:18.212922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:18.210664 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:19.167440 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:19.167399 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:19.176623 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:19.176590 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Oct 30 20:08:19.176764 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:19.176649 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Oct 30 20:08:19.210919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:19.209480 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:19.210919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:19.209567 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:19.413562 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:19.413523 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:19.707919 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:19.700475 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:20.189923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:20.175769 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Oct 30 20:08:20.189923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:20.175829 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Oct 30 20:08:20.221741 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:20.219496 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:20.221741 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:20.219558 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:21.207923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:21.183889 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Oct 30 20:08:21.207923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:21.198010 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Oct 30 20:08:21.211265 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:21.211228 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:21.211407 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:21.211288 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:21.330031 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:21.329979 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:21.463101 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:21.462993 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:21.907928 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:21.903262 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:22.137924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.135562 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:22.228990 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.228030 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/health\": context deadline exceeded" Oct 30 20:08:22.228990 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.228762 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Oct 30 20:08:22.228990 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.228801 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Oct 30 20:08:22.353923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.331049 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded" Oct 30 20:08:22.353923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.331646 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:08:22.480980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.464996 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:22.480980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.465570 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:08:22.760966 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.754159 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:22.946974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.946458 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:22.946974 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:22.946523 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:23.002420 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:23.002342 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:23.374925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:23.367141 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:23.374925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:23.367221 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:23.396923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:23.392182 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:23.491324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:23.491269 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:23.632995 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:23.628866 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:24.352926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.340875 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj"] Oct 30 20:08:24.354026 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.353986 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" containerID="cri-o://27b2c4d5c96c478e2ecf20505d4884fd0acf89a05de4e71120297257b6ac261d" gracePeriod=30 Oct 30 20:08:24.374925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.355861 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:24.426140 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.426102 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:24.447453 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.447413 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:24.550945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.545422 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:24.550945 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.545539 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:24.563987 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.552701 6411 patch_prober.go:29] interesting pod/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:24.618983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.614216 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:24.618983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.445785 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:24.618983 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.557050 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:24.633545 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.633511 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz"] Oct 30 20:08:24.634112 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.634077 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" containerID="cri-o://f0c413698473d8574218c1a2799e6f7ac85f978b2b669674d4814836a17e13ab" gracePeriod=30 Oct 30 20:08:24.710559 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.710523 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:24.820417 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.820380 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd"] Oct 30 20:08:24.871982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.856158 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" containerID="cri-o://b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf" gracePeriod=30 Oct 30 20:08:24.872624 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.872598 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2"] Oct 30 20:08:24.873226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.873189 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" containerID="cri-o://10c655f4708918b7579b9eaf436248b2c9cbbe159d86220ca8845a4520dba3e3" gracePeriod=30 Oct 30 20:08:24.893922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.877481 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:24.893922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.877534 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:24.996351 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.996307 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv"] Oct 30 20:08:25.014925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:24.996885 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" containerID="cri-o://474c3aba2c753f94581002778892d7ccd031db75ed1ea22eb4cd924ae98d5c12" gracePeriod=30 Oct 30 20:08:25.115538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.115508 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5"] Oct 30 20:08:25.116152 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.116120 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" containerID="cri-o://3f412cd1a29e5cdaf2eff2c4411aed99a1c22f40f3234ea844f0de5d8e4d6de4" gracePeriod=30 Oct 30 20:08:25.218955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.215491 6411 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": context deadline exceeded" Oct 30 20:08:25.353924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.349576 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:25.414980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.389986 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:25.423347 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.423309 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:08:25.510538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.510504 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:25.511163 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.511135 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:25.517416 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.517392 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:08:25.556057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.556020 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:25.556324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:25.556286 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:26.153922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:26.147806 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:26.516228 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:26.514074 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:26.516228 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:26.514614 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:26.629924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:26.616032 6411 patch_prober.go:29] interesting pod/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:08:26.629924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:26.616109 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:08:27.149935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:27.146303 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:27.176339 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:27.176298 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:27.211932 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:27.210006 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:27.221924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:27.220704 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:27.284778 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:27.284751 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:08:27.366962 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:27.366883 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:27.466656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:27.466575 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:27.472335 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:27.471826 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:08:28.335042 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:28.335010 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:29.903331 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:29.903280 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:30.307640 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.307603 6411 generic.go:356] "Generic (PLEG): container finished" podID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerID="f0c413698473d8574218c1a2799e6f7ac85f978b2b669674d4814836a17e13ab" exitCode=0 Oct 30 20:08:30.307777 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.307734 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" event={"ID":"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be","Type":"ContainerDied","Data":"f0c413698473d8574218c1a2799e6f7ac85f978b2b669674d4814836a17e13ab"} Oct 30 20:08:30.312497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.309818 6411 generic.go:356] "Generic (PLEG): container finished" podID="c7401270-59d3-4f59-af0c-c7943dde243b" containerID="27b2c4d5c96c478e2ecf20505d4884fd0acf89a05de4e71120297257b6ac261d" exitCode=0 Oct 30 20:08:30.312497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.309988 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" event={"ID":"c7401270-59d3-4f59-af0c-c7943dde243b","Type":"ContainerDied","Data":"27b2c4d5c96c478e2ecf20505d4884fd0acf89a05de4e71120297257b6ac261d"} Oct 30 20:08:30.312497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.311734 6411 generic.go:356] "Generic (PLEG): container finished" podID="860bfb55-555b-4198-b9ae-23712a7453e1" containerID="3f412cd1a29e5cdaf2eff2c4411aed99a1c22f40f3234ea844f0de5d8e4d6de4" exitCode=0 Oct 30 20:08:30.312497 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.311803 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" event={"ID":"860bfb55-555b-4198-b9ae-23712a7453e1","Type":"ContainerDied","Data":"3f412cd1a29e5cdaf2eff2c4411aed99a1c22f40f3234ea844f0de5d8e4d6de4"} Oct 30 20:08:30.317671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.317218 6411 generic.go:356] "Generic (PLEG): container finished" podID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerID="474c3aba2c753f94581002778892d7ccd031db75ed1ea22eb4cd924ae98d5c12" exitCode=0 Oct 30 20:08:30.317671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.317345 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" event={"ID":"708fc137-602b-42fc-9572-7aacdbffa8cb","Type":"ContainerDied","Data":"474c3aba2c753f94581002778892d7ccd031db75ed1ea22eb4cd924ae98d5c12"} Oct 30 20:08:30.322109 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.322058 6411 generic.go:356] "Generic (PLEG): container finished" podID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerID="10c655f4708918b7579b9eaf436248b2c9cbbe159d86220ca8845a4520dba3e3" exitCode=0 Oct 30 20:08:30.322217 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.322108 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" event={"ID":"b71651b4-40f9-4900-aa6d-84328edffa6d","Type":"ContainerDied","Data":"10c655f4708918b7579b9eaf436248b2c9cbbe159d86220ca8845a4520dba3e3"} Oct 30 20:08:30.322217 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.322160 6411 scope.go:122] "RemoveContainer" containerID="c2318339e9b73f9893c7f24eb4e1d5416cca1a3d3f3e93ee6cf4efa9b38a9598" Oct 30 20:08:30.350348 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.349997 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:08:30.393409 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.393289 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:08:30.396544 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.396522 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:08:30.405186 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.405153 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407648 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-kube-api-access-54zvr\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-kube-api-access-54zvr\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407686 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-token\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-token\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407716 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-token\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-token\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407775 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-podinfo\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407803 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-socket\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407848 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-podinfo\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407918 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-envoy\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407951 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-data\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.407989 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/708fc137-602b-42fc-9572-7aacdbffa8cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/708fc137-602b-42fc-9572-7aacdbffa8cb-istiod-ca-cert\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408026 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-socket\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408079 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b71651b4-40f9-4900-aa6d-84328edffa6d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71651b4-40f9-4900-aa6d-84328edffa6d-istiod-ca-cert\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408112 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-credential-socket\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408220 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-certs\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408256 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-socket\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408292 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-envoy\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408367 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-kube-api-access-97zn5\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-kube-api-access-97zn5\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408406 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-envoy\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408444 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-certs\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408482 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-certs\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408510 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-credential-socket\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408541 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-data\") pod \"b71651b4-40f9-4900-aa6d-84328edffa6d\" (UID: \"b71651b4-40f9-4900-aa6d-84328edffa6d\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408612 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-credential-socket\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408643 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-data\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408672 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-kube-api-access-2ks88\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-kube-api-access-2ks88\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408710 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istiod-ca-cert\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408740 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-podinfo\") pod \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\" (UID: \"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.408773 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-token\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-token\") pod \"708fc137-602b-42fc-9572-7aacdbffa8cb\" (UID: \"708fc137-602b-42fc-9572-7aacdbffa8cb\") " Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.409145 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/708fc137-602b-42fc-9572-7aacdbffa8cb-istiod-ca-cert" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.409152 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-certs" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.409402 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-socket" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.409652 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.409668 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/708fc137-602b-42fc-9572-7aacdbffa8cb-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.409678 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.409688 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b71651b4-40f9-4900-aa6d-84328edffa6d-istiod-ca-cert" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.409852 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-socket" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.410000 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-credential-socket" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.410767 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-credential-socket" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.410943 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-certs" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.411256 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-credential-socket" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.412738 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-data" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.414466 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-certs" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.416803 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istiod-ca-cert" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:30.418688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.418326 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-data" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.421587 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.421523 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-socket" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.423622 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.422512 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-token" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.423622 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.423578 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-data" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.425196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.425170 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-envoy" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.427256 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.427231 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-token" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.427256 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.427241 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-envoy" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.427475 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.427375 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-kube-api-access-97zn5" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "kube-api-access-97zn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.427475 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.427430 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-kube-api-access-2ks88" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "kube-api-access-2ks88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.427638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.427543 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-envoy" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.427638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.427630 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-token" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.427788 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.427678 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-kube-api-access-54zvr" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "kube-api-access-54zvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.428010 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.427987 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-podinfo" pod "b71651b4-40f9-4900-aa6d-84328edffa6d" (UID: "b71651b4-40f9-4900-aa6d-84328edffa6d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:30.428822 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.428790 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-podinfo" pod "708fc137-602b-42fc-9572-7aacdbffa8cb" (UID: "708fc137-602b-42fc-9572-7aacdbffa8cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:30.429038 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.429015 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-podinfo" pod "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" (UID: "20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.512837 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.512880 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.512941 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.512957 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54zvr\" (UniqueName: \"kubernetes.io/projected/708fc137-602b-42fc-9572-7aacdbffa8cb-kube-api-access-54zvr\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.512976 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.512989 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513002 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513015 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513028 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513041 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513054 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513067 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71651b4-40f9-4900-aa6d-84328edffa6d-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513079 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513091 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513105 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513119 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513133 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97zn5\" (UniqueName: \"kubernetes.io/projected/b71651b4-40f9-4900-aa6d-84328edffa6d-kube-api-access-97zn5\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513146 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513160 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513172 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/708fc137-602b-42fc-9572-7aacdbffa8cb-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513184 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71651b4-40f9-4900-aa6d-84328edffa6d-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513197 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513210 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.528553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.513223 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ks88\" (UniqueName: \"kubernetes.io/projected/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be-kube-api-access-2ks88\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.546391 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.546363 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:08:30.614391 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614352 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-socket\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.614391 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614402 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-credential-socket\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.614656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614438 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-istio-token\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-istio-token\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.614656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614480 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-kube-api-access-hdk4x\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-kube-api-access-hdk4x\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.614656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614521 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7401270-59d3-4f59-af0c-c7943dde243b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7401270-59d3-4f59-af0c-c7943dde243b-istiod-ca-cert\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.614656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614583 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-envoy\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.614656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614646 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c7401270-59d3-4f59-af0c-c7943dde243b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7401270-59d3-4f59-af0c-c7943dde243b-istio-podinfo\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.614930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614676 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-socket" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.614930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614775 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-certs\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.614930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.614807 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-data\") pod \"c7401270-59d3-4f59-af0c-c7943dde243b\" (UID: \"c7401270-59d3-4f59-af0c-c7943dde243b\") " Oct 30 20:08:30.615101 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.615056 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7401270-59d3-4f59-af0c-c7943dde243b-istiod-ca-cert" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:30.615297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.615273 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.615297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.615299 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7401270-59d3-4f59-af0c-c7943dde243b-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.615483 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.615331 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-certs" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.615535 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.615492 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-data" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.615982 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.615807 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-credential-socket" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.618085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.617988 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-envoy" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.618085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.618065 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-kube-api-access-hdk4x" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "kube-api-access-hdk4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.618085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.618069 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7401270-59d3-4f59-af0c-c7943dde243b-istio-podinfo" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:30.619964 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.619936 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-istio-token" pod "c7401270-59d3-4f59-af0c-c7943dde243b" (UID: "c7401270-59d3-4f59-af0c-c7943dde243b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.716562 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.716529 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.716562 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.716560 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.716562 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.716568 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.716872 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.716578 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.716872 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.716587 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdk4x\" (UniqueName: \"kubernetes.io/projected/c7401270-59d3-4f59-af0c-c7943dde243b-kube-api-access-hdk4x\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.716872 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.716595 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7401270-59d3-4f59-af0c-c7943dde243b-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.716872 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.716604 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7401270-59d3-4f59-af0c-c7943dde243b-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.837674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.837596 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:08:30.906716 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.906682 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:08:30.918278 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918246 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-podinfo\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.918456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918300 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-data\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.918456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918336 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-token\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-token\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.918456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918365 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-kube-api-access-84b85\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-kube-api-access-84b85\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.918456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918396 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/860bfb55-555b-4198-b9ae-23712a7453e1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/860bfb55-555b-4198-b9ae-23712a7453e1-istiod-ca-cert\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.918456 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918432 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-socket\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.918723 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918459 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-kube-api-access-khn8r\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-kube-api-access-khn8r\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.918723 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918504 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/860bfb55-555b-4198-b9ae-23712a7453e1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/860bfb55-555b-4198-b9ae-23712a7453e1-istio-podinfo\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.919255 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.918936 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/860bfb55-555b-4198-b9ae-23712a7453e1-istiod-ca-cert" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:30.919255 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.919217 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-data" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.919455 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.919123 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-socket" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.919514 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.919453 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-certs\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.919807 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.919773 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-certs" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.920511 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.919964 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-credential-socket\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.920511 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.920203 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-credential-socket" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.920748 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.920676 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-envoy\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.921179 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.921140 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-certs" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.921361 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.921290 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-certs\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.922029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.921402 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-socket\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.922029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.921507 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-istio-token\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-istio-token\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.922029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.921552 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-credential-socket\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.922029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.921578 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-envoy\") pod \"860bfb55-555b-4198-b9ae-23712a7453e1\" (UID: \"860bfb55-555b-4198-b9ae-23712a7453e1\") " Oct 30 20:08:30.922029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.921640 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85e7c420-1e78-42cf-abc6-25366f1e9387-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85e7c420-1e78-42cf-abc6-25366f1e9387-istiod-ca-cert\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.922029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.921670 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-data\") pod \"85e7c420-1e78-42cf-abc6-25366f1e9387\" (UID: \"85e7c420-1e78-42cf-abc6-25366f1e9387\") " Oct 30 20:08:30.922390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.922231 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.922390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.922251 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.922390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.922265 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.922390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.922279 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.922390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.922293 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/860bfb55-555b-4198-b9ae-23712a7453e1-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.922390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.922307 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:30.923921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.923834 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-envoy" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.923921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.923851 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85e7c420-1e78-42cf-abc6-25366f1e9387-istiod-ca-cert" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:08:30.924258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.924109 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-socket" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.924258 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.924165 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-data" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.924486 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.924456 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-credential-socket" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.925374 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.925345 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/860bfb55-555b-4198-b9ae-23712a7453e1-istio-podinfo" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:30.926184 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.926146 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-podinfo" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:08:30.926184 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.926164 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-kube-api-access-khn8r" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "kube-api-access-khn8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.926348 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.926182 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-token" pod "85e7c420-1e78-42cf-abc6-25366f1e9387" (UID: "85e7c420-1e78-42cf-abc6-25366f1e9387"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.926529 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.926501 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-istio-token" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:30.927789 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.927762 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-envoy" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:08:30.928503 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:30.928475 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-kube-api-access-84b85" pod "860bfb55-555b-4198-b9ae-23712a7453e1" (UID: "860bfb55-555b-4198-b9ae-23712a7453e1"). InnerVolumeSpecName "kube-api-access-84b85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:08:31.023560 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023459 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023560 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023494 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85e7c420-1e78-42cf-abc6-25366f1e9387-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023560 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023510 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023560 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023524 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023560 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023537 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023560 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023551 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84b85\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-kube-api-access-84b85\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023564 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khn8r\" (UniqueName: \"kubernetes.io/projected/85e7c420-1e78-42cf-abc6-25366f1e9387-kube-api-access-khn8r\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023577 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/860bfb55-555b-4198-b9ae-23712a7453e1-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023592 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023605 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/860bfb55-555b-4198-b9ae-23712a7453e1-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023618 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/860bfb55-555b-4198-b9ae-23712a7453e1-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.023849 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.023631 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e7c420-1e78-42cf-abc6-25366f1e9387-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:08:31.121822 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.121785 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:31.330671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.330589 6411 generic.go:356] "Generic (PLEG): container finished" podID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerID="b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf" exitCode=0 Oct 30 20:08:31.330671 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.330648 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" Oct 30 20:08:31.330892 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.330686 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" event={"ID":"85e7c420-1e78-42cf-abc6-25366f1e9387","Type":"ContainerDied","Data":"b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf"} Oct 30 20:08:31.330892 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.330752 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd" event={"ID":"85e7c420-1e78-42cf-abc6-25366f1e9387","Type":"ContainerDied","Data":"0a5d7973129f939ac66db27f751abedd8fa30fd0315e54587c6fa3420451ef25"} Oct 30 20:08:31.330892 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.330769 6411 scope.go:122] "RemoveContainer" containerID="b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf" Oct 30 20:08:31.332573 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.332235 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" Oct 30 20:08:31.332573 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.332479 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5" event={"ID":"860bfb55-555b-4198-b9ae-23712a7453e1","Type":"ContainerDied","Data":"6c8c8175a04d899cbe09cbb4f1d28dbc7ca1c897d08b7814b4398d0853b87122"} Oct 30 20:08:31.333881 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.333842 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" Oct 30 20:08:31.334017 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.333880 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv" event={"ID":"708fc137-602b-42fc-9572-7aacdbffa8cb","Type":"ContainerDied","Data":"25e55960b70cbc246edec000a473bfff08ff9d72e9c3b6696d585a43f143193a"} Oct 30 20:08:31.336988 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.336971 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" Oct 30 20:08:31.337118 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.337016 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2" event={"ID":"b71651b4-40f9-4900-aa6d-84328edffa6d","Type":"ContainerDied","Data":"ab90c0d42d3adb6beba7a7b4e90d55069eaf77d0af70ef3c5b9713d5d208af92"} Oct 30 20:08:31.339423 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.339396 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" event={"ID":"20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be","Type":"ContainerDied","Data":"181aa933ee9a75a833202d83a63baac372663c7a9e050eda05bf7d640ba78826"} Oct 30 20:08:31.339572 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.339422 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz" Oct 30 20:08:31.340953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.340884 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" event={"ID":"c7401270-59d3-4f59-af0c-c7943dde243b","Type":"ContainerDied","Data":"37627afce21a926d7964686c2bf264fc6ac4b7cd34204716cdeddce8d9340d5a"} Oct 30 20:08:31.340953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.340929 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj" Oct 30 20:08:31.344041 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.344019 6411 scope.go:122] "RemoveContainer" containerID="868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01" Oct 30 20:08:31.356565 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.356465 6411 scope.go:122] "RemoveContainer" containerID="b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf" Oct 30 20:08:31.356921 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:31.356879 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf\": container with ID starting with b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf not found: ID does not exist" containerID="b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf" Oct 30 20:08:31.357012 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.356932 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf"} err="failed to get container status \"b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf\": rpc error: code = NotFound desc = could not find container \"b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf\": container with ID starting with b7ab8f174dc8c8862e5d83053f4e10e33f241e5b09733666f08669ed8b2a94bf not found: ID does not exist" Oct 30 20:08:31.357012 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.356955 6411 scope.go:122] "RemoveContainer" containerID="868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01" Oct 30 20:08:31.357276 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:31.357253 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01\": container with ID starting with 868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01 not found: ID does not exist" containerID="868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01" Oct 30 20:08:31.357337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.357282 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01"} err="failed to get container status \"868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01\": rpc error: code = NotFound desc = could not find container \"868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01\": container with ID starting with 868746af1cc120bac4a81fdfe41044d5661bad974e90cea337131873d32fdc01 not found: ID does not exist" Oct 30 20:08:31.357337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.357296 6411 scope.go:122] "RemoveContainer" containerID="3f412cd1a29e5cdaf2eff2c4411aed99a1c22f40f3234ea844f0de5d8e4d6de4" Oct 30 20:08:31.365768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.365752 6411 scope.go:122] "RemoveContainer" containerID="474c3aba2c753f94581002778892d7ccd031db75ed1ea22eb4cd924ae98d5c12" Oct 30 20:08:31.374435 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.374416 6411 scope.go:122] "RemoveContainer" containerID="10c655f4708918b7579b9eaf436248b2c9cbbe159d86220ca8845a4520dba3e3" Oct 30 20:08:31.383985 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.383911 6411 scope.go:122] "RemoveContainer" containerID="f0c413698473d8574218c1a2799e6f7ac85f978b2b669674d4814836a17e13ab" Oct 30 20:08:31.392393 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.392376 6411 scope.go:122] "RemoveContainer" containerID="27b2c4d5c96c478e2ecf20505d4884fd0acf89a05de4e71120297257b6ac261d" Oct 30 20:08:31.469969 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.469931 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5"] Oct 30 20:08:31.482570 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.482541 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-knpq5"] Oct 30 20:08:31.515713 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.515678 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd"] Oct 30 20:08:31.521439 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.521410 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-j5zcd"] Oct 30 20:08:31.559256 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.559218 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv"] Oct 30 20:08:31.567022 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.566997 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-v9plv"] Oct 30 20:08:31.604834 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.604761 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj"] Oct 30 20:08:31.611085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.611059 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-vhlmj"] Oct 30 20:08:31.648992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.648956 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz"] Oct 30 20:08:31.656966 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.656930 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f10d2-openshift-default-6c6777d6f5-l9vxz"] Oct 30 20:08:31.683477 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.683447 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:31.694370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.694346 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2"] Oct 30 20:08:31.700740 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.700718 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-62f4a1-openshift-default-7fc84fbccc-hwcg2"] Oct 30 20:08:31.732165 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.732138 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" path="/var/lib/kubelet/pods/20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be/volumes" Oct 30 20:08:31.732624 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.732610 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" path="/var/lib/kubelet/pods/708fc137-602b-42fc-9572-7aacdbffa8cb/volumes" Oct 30 20:08:31.733022 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.733009 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" path="/var/lib/kubelet/pods/85e7c420-1e78-42cf-abc6-25366f1e9387/volumes" Oct 30 20:08:31.733698 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.733685 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" path="/var/lib/kubelet/pods/860bfb55-555b-4198-b9ae-23712a7453e1/volumes" Oct 30 20:08:31.734196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.734180 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" path="/var/lib/kubelet/pods/b71651b4-40f9-4900-aa6d-84328edffa6d/volumes" Oct 30 20:08:31.734588 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:31.734574 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" path="/var/lib/kubelet/pods/c7401270-59d3-4f59-af0c-c7943dde243b/volumes" Oct 30 20:08:32.307128 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:32.307094 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:34.142938 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:34.142885 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:35.093058 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:08:35.093023 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:08:35.244915 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:35.244871 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:36.062716 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:36.062678 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:37.669615 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:37.669582 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:38.024585 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:38.024494 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:39.005873 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:39.005839 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:40.251007 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:40.248252 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:40.453926 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:40.453872 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:42.511807 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:42.511768 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:44.415960 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:44.415888 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:45.509262 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:45.509227 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:46.359752 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:46.359697 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:48.459108 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:48.459048 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:50.400596 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:50.400564 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:51.206682 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:51.206646 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:51.408144 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:51.408110 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:52.425422 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:52.425389 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:54.687920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:54.685765 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:56.287492 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:56.287461 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx"] Oct 30 20:08:56.306482 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:56.306451 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs"] Oct 30 20:08:56.402888 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:56.402851 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:56.616571 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:56.616488 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:08:58.406410 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:08:58.406372 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:00.247718 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:00.247672 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:02.118148 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:02.118110 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:02.303315 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:02.303280 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:05.097121 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:05.097077 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:09:08.838742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:08.838702 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:09.362927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:09.358339 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:09.563922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:09.563604 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:10.034366 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:10.017756 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:11.314357 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.314319 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" containerID="cri-o://5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765" gracePeriod=45 Oct 30 20:09:11.329408 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.329357 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" containerID="cri-o://551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844" gracePeriod=45 Oct 30 20:09:11.603778 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.603751 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:09:11.631325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.631295 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:09:11.700075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.699783 6411 generic.go:356] "Generic (PLEG): container finished" podID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerID="551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844" exitCode=0 Oct 30 20:09:11.700075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.699833 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" Oct 30 20:09:11.700075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.699854 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" event={"ID":"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c","Type":"ContainerDied","Data":"551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844"} Oct 30 20:09:11.700075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.699923 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" event={"ID":"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c","Type":"ContainerDied","Data":"ae72b4ee386bdd442202d32bc027f023e89d682cc16955ff4a2a39a7bfaf23ca"} Oct 30 20:09:11.700075 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.699949 6411 scope.go:122] "RemoveContainer" containerID="551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844" Oct 30 20:09:11.702130 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.702106 6411 generic.go:356] "Generic (PLEG): container finished" podID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerID="5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765" exitCode=0 Oct 30 20:09:11.702278 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.702141 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" event={"ID":"0b7d73fb-1fb8-4969-a783-f242346001f2","Type":"ContainerDied","Data":"5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765"} Oct 30 20:09:11.702278 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.702152 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" Oct 30 20:09:11.702278 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.702171 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" event={"ID":"0b7d73fb-1fb8-4969-a783-f242346001f2","Type":"ContainerDied","Data":"bf16c37ab61c235ec4d71cde81b063e9de0788afd903448d0ec9c4856e6ff239"} Oct 30 20:09:11.709707 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.709677 6411 scope.go:122] "RemoveContainer" containerID="551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844" Oct 30 20:09:11.710013 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:11.709994 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844\": container with ID starting with 551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844 not found: ID does not exist" containerID="551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844" Oct 30 20:09:11.710088 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.710020 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844"} err="failed to get container status \"551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844\": rpc error: code = NotFound desc = could not find container \"551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844\": container with ID starting with 551278e099fd72d3ce7ff1887a1a6f80c42db65c51f3ad123fbbb31400c42844 not found: ID does not exist" Oct 30 20:09:11.710088 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.710037 6411 scope.go:122] "RemoveContainer" containerID="5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765" Oct 30 20:09:11.720013 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.719995 6411 scope.go:122] "RemoveContainer" containerID="5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765" Oct 30 20:09:11.720295 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:11.720272 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765\": container with ID starting with 5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765 not found: ID does not exist" containerID="5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765" Oct 30 20:09:11.720366 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.720305 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765"} err="failed to get container status \"5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765\": rpc error: code = NotFound desc = could not find container \"5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765\": container with ID starting with 5512d21ec258c76a430fc104376c10314a21c0f9bdd07eec8406f2e1581a3765 not found: ID does not exist" Oct 30 20:09:11.730383 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.730356 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-tmp-dir\") pod \"0b7d73fb-1fb8-4969-a783-f242346001f2\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " Oct 30 20:09:11.730504 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.730419 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-kube-api-access-hjdrv\" (UniqueName: \"kubernetes.io/projected/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-kube-api-access-hjdrv\") pod \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " Oct 30 20:09:11.730504 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.730469 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b7d73fb-1fb8-4969-a783-f242346001f2-kube-api-access-hcwb2\" (UniqueName: \"kubernetes.io/projected/0b7d73fb-1fb8-4969-a783-f242346001f2-kube-api-access-hcwb2\") pod \"0b7d73fb-1fb8-4969-a783-f242346001f2\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " Oct 30 20:09:11.730643 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.730505 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-model-cache\") pod \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " Oct 30 20:09:11.730643 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.730554 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-home\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-home\") pod \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " Oct 30 20:09:11.730643 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.730623 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-dshm\") pod \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " Oct 30 20:09:11.730811 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.730672 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-tmp-dir" pod "0b7d73fb-1fb8-4969-a783-f242346001f2" (UID: "0b7d73fb-1fb8-4969-a783-f242346001f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:11.731959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731044 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tmp-dir\") pod \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " Oct 30 20:09:11.731959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731135 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-model-cache\") pod \"0b7d73fb-1fb8-4969-a783-f242346001f2\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " Oct 30 20:09:11.731959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731163 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-dshm\") pod \"0b7d73fb-1fb8-4969-a783-f242346001f2\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " Oct 30 20:09:11.731959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731304 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tls-certs\") pod \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\" (UID: \"d6a26c55-a188-4bd8-ba15-60bb3a5deb2c\") " Oct 30 20:09:11.731959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731317 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tmp-dir" pod "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" (UID: "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:11.731959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731339 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b7d73fb-1fb8-4969-a783-f242346001f2-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7d73fb-1fb8-4969-a783-f242346001f2-tls-certs\") pod \"0b7d73fb-1fb8-4969-a783-f242346001f2\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " Oct 30 20:09:11.731959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731400 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-home\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-home\") pod \"0b7d73fb-1fb8-4969-a783-f242346001f2\" (UID: \"0b7d73fb-1fb8-4969-a783-f242346001f2\") " Oct 30 20:09:11.731959 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731566 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-model-cache" pod "0b7d73fb-1fb8-4969-a783-f242346001f2" (UID: "0b7d73fb-1fb8-4969-a783-f242346001f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:11.732512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731968 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.732512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.731989 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.732512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.732002 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.732512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.732276 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-home" pod "0b7d73fb-1fb8-4969-a783-f242346001f2" (UID: "0b7d73fb-1fb8-4969-a783-f242346001f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:11.733020 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.732999 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-home" pod "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" (UID: "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:11.733442 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.733422 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-model-cache" pod "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" (UID: "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:11.736878 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.736848 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7d73fb-1fb8-4969-a783-f242346001f2-kube-api-access-hcwb2" pod "0b7d73fb-1fb8-4969-a783-f242346001f2" (UID: "0b7d73fb-1fb8-4969-a783-f242346001f2"). InnerVolumeSpecName "kube-api-access-hcwb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:11.737084 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.736883 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-dshm" pod "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" (UID: "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:11.737084 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.737003 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-dshm" pod "0b7d73fb-1fb8-4969-a783-f242346001f2" (UID: "0b7d73fb-1fb8-4969-a783-f242346001f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:11.737534 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.737505 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tls-certs" pod "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" (UID: "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:09:11.737603 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.737511 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b7d73fb-1fb8-4969-a783-f242346001f2-tls-certs" pod "0b7d73fb-1fb8-4969-a783-f242346001f2" (UID: "0b7d73fb-1fb8-4969-a783-f242346001f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:09:11.738112 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.738085 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-kube-api-access-hjdrv" pod "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" (UID: "d6a26c55-a188-4bd8-ba15-60bb3a5deb2c"). InnerVolumeSpecName "kube-api-access-hjdrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:11.759305 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.759278 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:11.833530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833461 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.833530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833488 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.833530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833499 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7d73fb-1fb8-4969-a783-f242346001f2-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.833530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833506 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7d73fb-1fb8-4969-a783-f242346001f2-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.833530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833515 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjdrv\" (UniqueName: \"kubernetes.io/projected/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-kube-api-access-hjdrv\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.833530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833524 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcwb2\" (UniqueName: \"kubernetes.io/projected/0b7d73fb-1fb8-4969-a783-f242346001f2-kube-api-access-hcwb2\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.833530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833534 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.833840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833542 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:11.833840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:11.833549 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:12.045414 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:12.045383 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx"] Oct 30 20:09:12.053691 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:12.053666 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx"] Oct 30 20:09:12.100765 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:12.100702 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs"] Oct 30 20:09:12.103555 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:12.103532 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs"] Oct 30 20:09:12.518323 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:12.518287 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/distinct-w2-efbac3-kserve-5dbdfdd9bc-x7xdx" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:09:12.616148 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:12.616113 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-efbac3/simulator-w2-efbac3-kserve-855cc55c8f-st9cs" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded" Oct 30 20:09:13.733361 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:13.733327 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" path="/var/lib/kubelet/pods/0b7d73fb-1fb8-4969-a783-f242346001f2/volumes" Oct 30 20:09:13.733739 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:13.733730 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" path="/var/lib/kubelet/pods/d6a26c55-a188-4bd8-ba15-60bb3a5deb2c/volumes" Oct 30 20:09:16.204737 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:16.204697 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:16.992266 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:16.992237 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr"] Oct 30 20:09:16.992508 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:16.992462 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" podUID="b0c2935f-803d-4343-94a7-af159d12e098" containerName="maas-api" containerID="cri-o://4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359" gracePeriod=30 Oct 30 20:09:17.084889 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.084856 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:17.256062 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.256000 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:09:17.324946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.312684 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-maas-api-tls\") pod \"b0c2935f-803d-4343-94a7-af159d12e098\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " Oct 30 20:09:17.324946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.312860 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0c2935f-803d-4343-94a7-af159d12e098-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c2935f-803d-4343-94a7-af159d12e098-openshift-service-ca\") pod \"b0c2935f-803d-4343-94a7-af159d12e098\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " Oct 30 20:09:17.324946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.312964 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0c2935f-803d-4343-94a7-af159d12e098-kube-api-access-slrfh\" (UniqueName: \"kubernetes.io/projected/b0c2935f-803d-4343-94a7-af159d12e098-kube-api-access-slrfh\") pod \"b0c2935f-803d-4343-94a7-af159d12e098\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " Oct 30 20:09:17.324946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.313006 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-metrics-certs\") pod \"b0c2935f-803d-4343-94a7-af159d12e098\" (UID: \"b0c2935f-803d-4343-94a7-af159d12e098\") " Oct 30 20:09:17.324946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.317437 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0c2935f-803d-4343-94a7-af159d12e098-openshift-service-ca" pod "b0c2935f-803d-4343-94a7-af159d12e098" (UID: "b0c2935f-803d-4343-94a7-af159d12e098"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:09:17.324946 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.320320 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-metrics-certs" pod "b0c2935f-803d-4343-94a7-af159d12e098" (UID: "b0c2935f-803d-4343-94a7-af159d12e098"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:09:17.349674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.349487 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0c2935f-803d-4343-94a7-af159d12e098-kube-api-access-slrfh" pod "b0c2935f-803d-4343-94a7-af159d12e098" (UID: "b0c2935f-803d-4343-94a7-af159d12e098"). InnerVolumeSpecName "kube-api-access-slrfh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:17.349674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.349585 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-maas-api-tls" pod "b0c2935f-803d-4343-94a7-af159d12e098" (UID: "b0c2935f-803d-4343-94a7-af159d12e098"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:09:17.414604 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.414575 6411 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c2935f-803d-4343-94a7-af159d12e098-openshift-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:17.414604 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.414600 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slrfh\" (UniqueName: \"kubernetes.io/projected/b0c2935f-803d-4343-94a7-af159d12e098-kube-api-access-slrfh\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:17.414604 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.414613 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:17.414604 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.414623 6411 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c2935f-803d-4343-94a7-af159d12e098-maas-api-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:17.667612 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.667560 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:17.739342 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.739308 6411 generic.go:356] "Generic (PLEG): container finished" podID="b0c2935f-803d-4343-94a7-af159d12e098" containerID="4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359" exitCode=0 Oct 30 20:09:17.739491 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.739354 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" Oct 30 20:09:17.739491 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.739395 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" event={"ID":"b0c2935f-803d-4343-94a7-af159d12e098","Type":"ContainerDied","Data":"4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359"} Oct 30 20:09:17.739491 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.739448 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr" event={"ID":"b0c2935f-803d-4343-94a7-af159d12e098","Type":"ContainerDied","Data":"1f2583b6cb32736bc07d65d1910ede77b338556d9b292fc204409eead7bd7b54"} Oct 30 20:09:17.739491 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.739476 6411 scope.go:122] "RemoveContainer" containerID="4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359" Oct 30 20:09:17.749826 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.749803 6411 scope.go:122] "RemoveContainer" containerID="4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359" Oct 30 20:09:17.750144 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:17.750120 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359\": container with ID starting with 4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359 not found: ID does not exist" containerID="4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359" Oct 30 20:09:17.750231 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.750153 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359"} err="failed to get container status \"4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359\": rpc error: code = NotFound desc = could not find container \"4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359\": container with ID starting with 4036c4f1efcd095c5c7cb6a4164f0103422e8dea246d0e8bd08b2a4a8268d359 not found: ID does not exist" Oct 30 20:09:17.802406 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.802379 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr"] Oct 30 20:09:17.814730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.814709 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-efbac3-696c7c98b-x92rr"] Oct 30 20:09:17.934003 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:17.933867 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:18.382182 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:18.382128 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:19.315977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.315943 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:19.573554 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.573445 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:19.707527 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.706693 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9"] Oct 30 20:09:19.707527 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.706982 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" containerID="cri-o://320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9" gracePeriod=30 Oct 30 20:09:19.719140 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.718655 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr"] Oct 30 20:09:19.719140 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.718911 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" containerID="cri-o://17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf" gracePeriod=30 Oct 30 20:09:19.727751 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.725754 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl"] Oct 30 20:09:19.727751 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.725982 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" containerID="cri-o://a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e" gracePeriod=30 Oct 30 20:09:19.756250 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.755726 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0c2935f-803d-4343-94a7-af159d12e098" path="/var/lib/kubelet/pods/b0c2935f-803d-4343-94a7-af159d12e098/volumes" Oct 30 20:09:19.756736 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.756711 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt"] Oct 30 20:09:19.756929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.756888 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" containerID="cri-o://76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842" gracePeriod=30 Oct 30 20:09:19.914719 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:19.914672 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:23.031955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:23.031879 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:23.297356 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:23.297270 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:25.098918 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.092787 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:09:25.130759 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.130729 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:09:25.135454 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.135432 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:09:25.139496 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.139475 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:09:25.218821 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.218788 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-podinfo\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.218841 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-socket\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.218888 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-certs\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.218937 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-token\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-token\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.218968 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-envoy\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219007 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-kube-api-access-rrx6n\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-kube-api-access-rrx6n\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219037 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-kube-api-access-ljslf\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-kube-api-access-ljslf\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219067 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-podinfo\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219107 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istiod-ca-cert\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219139 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-data\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219176 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-envoy\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219221 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-data\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219260 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-credential-socket\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219286 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-token\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-token\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219318 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9733f9d7-b928-4bd7-be04-9ac327153c27-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9733f9d7-b928-4bd7-be04-9ac327153c27-istiod-ca-cert\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219359 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-podinfo\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219393 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-podinfo\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219427 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-envoy\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219457 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-socket\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219484 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-token\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-token\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219536 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-token\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-token\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219586 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-credential-socket\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219679 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-data\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219771 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-envoy\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219800 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-credential-socket\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219825 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-certs\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219854 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-socket\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219879 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-certs\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219950 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-certs\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.219985 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-credential-socket\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.220059 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istiod-ca-cert\") pod \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\" (UID: \"d0964117-5aaf-4b8d-b0b1-ffbfc4496522\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.220091 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istiod-ca-cert\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.220145 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-data\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.220178 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-kube-api-access-d8vkb\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-kube-api-access-d8vkb\") pod \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\" (UID: \"a210c8b6-cbb5-4ed7-b271-49e4912d42e6\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.220206 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-kube-api-access-m7fbn\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-kube-api-access-m7fbn\") pod \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\" (UID: \"546bae1b-d60e-47b5-8414-6fd7a0d66a5e\") " Oct 30 20:09:25.220512 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.220266 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-socket\") pod \"9733f9d7-b928-4bd7-be04-9ac327153c27\" (UID: \"9733f9d7-b928-4bd7-be04-9ac327153c27\") " Oct 30 20:09:25.222325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.220956 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-socket" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.222325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.221860 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istiod-ca-cert" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:09:25.222325 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.222171 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-socket" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.222544 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.222337 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-certs" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.222544 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.222525 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-socket" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.223630 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.223363 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-data" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.223630 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.223560 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istiod-ca-cert" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:09:25.223942 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.223828 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-certs" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.224021 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.223980 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-credential-socket" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.224066 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-credential-socket" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.224315 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istiod-ca-cert" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.225034 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9733f9d7-b928-4bd7-be04-9ac327153c27-istiod-ca-cert" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.225402 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-certs" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.225450 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-data" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.225601 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-credential-socket" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.225702 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-data" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.225785 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-kube-api-access-rrx6n" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "kube-api-access-rrx6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.226128 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-socket" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.227127 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-credential-socket" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.227446 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-data" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.228953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.228798 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-certs" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.229754 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.229324 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-token" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:25.232240 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.232175 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-podinfo" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:09:25.232353 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.232319 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-token" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:25.232467 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.232434 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-kube-api-access-d8vkb" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "kube-api-access-d8vkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:25.232591 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.232532 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-envoy" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.232670 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.232632 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-podinfo" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:09:25.233213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.233191 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-envoy" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.233494 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.233309 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-token" pod "d0964117-5aaf-4b8d-b0b1-ffbfc4496522" (UID: "d0964117-5aaf-4b8d-b0b1-ffbfc4496522"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:25.233494 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.233353 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-token" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:25.233494 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.233420 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-envoy" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.233494 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.233426 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-podinfo" pod "a210c8b6-cbb5-4ed7-b271-49e4912d42e6" (UID: "a210c8b6-cbb5-4ed7-b271-49e4912d42e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:09:25.233940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.233918 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-kube-api-access-m7fbn" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "kube-api-access-m7fbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:25.234187 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.234168 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-podinfo" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:09:25.234566 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.234531 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-kube-api-access-ljslf" pod "9733f9d7-b928-4bd7-be04-9ac327153c27" (UID: "9733f9d7-b928-4bd7-be04-9ac327153c27"). InnerVolumeSpecName "kube-api-access-ljslf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:25.235295 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.235269 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-envoy" pod "546bae1b-d60e-47b5-8414-6fd7a0d66a5e" (UID: "546bae1b-d60e-47b5-8414-6fd7a0d66a5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:25.322952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.322852 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.322952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.322889 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.322952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.322923 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.322952 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.322946 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.322960 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.322972 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrx6n\" (UniqueName: \"kubernetes.io/projected/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-kube-api-access-rrx6n\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.322987 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljslf\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-kube-api-access-ljslf\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323006 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323020 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323034 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323047 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323063 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323076 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323089 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323102 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9733f9d7-b928-4bd7-be04-9ac327153c27-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323118 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323132 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323145 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323160 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323176 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323191 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323203 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323215 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323232 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323244 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323256 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323269 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323287 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.323285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323301 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.324704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323315 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.324704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323332 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0964117-5aaf-4b8d-b0b1-ffbfc4496522-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.324704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323345 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.324704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323357 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.324704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323377 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8vkb\" (UniqueName: \"kubernetes.io/projected/a210c8b6-cbb5-4ed7-b271-49e4912d42e6-kube-api-access-d8vkb\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.324704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323395 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7fbn\" (UniqueName: \"kubernetes.io/projected/546bae1b-d60e-47b5-8414-6fd7a0d66a5e-kube-api-access-m7fbn\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.324704 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.323408 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9733f9d7-b928-4bd7-be04-9ac327153c27-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:25.788085 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.788057 6411 generic.go:356] "Generic (PLEG): container finished" podID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerID="320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9" exitCode=0 Oct 30 20:09:25.788239 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.788122 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" Oct 30 20:09:25.788239 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.788138 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" event={"ID":"546bae1b-d60e-47b5-8414-6fd7a0d66a5e","Type":"ContainerDied","Data":"320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9"} Oct 30 20:09:25.788239 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.788195 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9" event={"ID":"546bae1b-d60e-47b5-8414-6fd7a0d66a5e","Type":"ContainerDied","Data":"7c4119d1c8975313c993cfa47eea97f3e1ccf0e2938f87b0d7b0cfc48dcb4625"} Oct 30 20:09:25.788239 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.788225 6411 scope.go:122] "RemoveContainer" containerID="320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9" Oct 30 20:09:25.789569 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.789548 6411 generic.go:356] "Generic (PLEG): container finished" podID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerID="76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842" exitCode=0 Oct 30 20:09:25.789674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.789575 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" event={"ID":"d0964117-5aaf-4b8d-b0b1-ffbfc4496522","Type":"ContainerDied","Data":"76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842"} Oct 30 20:09:25.789674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.789627 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" event={"ID":"d0964117-5aaf-4b8d-b0b1-ffbfc4496522","Type":"ContainerDied","Data":"659f5900c8f8686d9a04a96418051d43d42cf2845d1a6554eaa02ff65dac95d2"} Oct 30 20:09:25.789674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.789628 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt" Oct 30 20:09:25.791283 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.791260 6411 generic.go:356] "Generic (PLEG): container finished" podID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerID="17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf" exitCode=0 Oct 30 20:09:25.791376 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.791337 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" Oct 30 20:09:25.791376 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.791346 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" event={"ID":"a210c8b6-cbb5-4ed7-b271-49e4912d42e6","Type":"ContainerDied","Data":"17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf"} Oct 30 20:09:25.791457 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.791383 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr" event={"ID":"a210c8b6-cbb5-4ed7-b271-49e4912d42e6","Type":"ContainerDied","Data":"afd6adbcc6adf89711a34ce5598dd341b470413d202db4a55e780dc53a104d30"} Oct 30 20:09:25.792914 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.792878 6411 generic.go:356] "Generic (PLEG): container finished" podID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerID="a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e" exitCode=0 Oct 30 20:09:25.793000 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.792926 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" event={"ID":"9733f9d7-b928-4bd7-be04-9ac327153c27","Type":"ContainerDied","Data":"a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e"} Oct 30 20:09:25.793000 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.792970 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" event={"ID":"9733f9d7-b928-4bd7-be04-9ac327153c27","Type":"ContainerDied","Data":"9e1979bb97529da97a7088754254c24f81ceb4c3e794eb01b5d02a5a1ca896f9"} Oct 30 20:09:25.793000 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.792986 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl" Oct 30 20:09:25.798992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.798973 6411 scope.go:122] "RemoveContainer" containerID="320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9" Oct 30 20:09:25.799250 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:25.799229 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9\": container with ID starting with 320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9 not found: ID does not exist" containerID="320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9" Oct 30 20:09:25.799337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.799260 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9"} err="failed to get container status \"320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9\": rpc error: code = NotFound desc = could not find container \"320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9\": container with ID starting with 320cae89a254f79fd7c1ea15ac070382bbaf46721c4859870e3f427f07083ab9 not found: ID does not exist" Oct 30 20:09:25.799337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.799282 6411 scope.go:122] "RemoveContainer" containerID="76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842" Oct 30 20:09:25.807342 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.807324 6411 scope.go:122] "RemoveContainer" containerID="76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842" Oct 30 20:09:25.807601 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:25.807580 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842\": container with ID starting with 76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842 not found: ID does not exist" containerID="76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842" Oct 30 20:09:25.807683 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.807609 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842"} err="failed to get container status \"76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842\": rpc error: code = NotFound desc = could not find container \"76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842\": container with ID starting with 76419bce64816ab679d4f73d014187d92881054449f144a2138ad188cfb36842 not found: ID does not exist" Oct 30 20:09:25.807683 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.807629 6411 scope.go:122] "RemoveContainer" containerID="17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf" Oct 30 20:09:25.815298 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.815270 6411 scope.go:122] "RemoveContainer" containerID="17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf" Oct 30 20:09:25.815888 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:25.815857 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf\": container with ID starting with 17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf not found: ID does not exist" containerID="17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf" Oct 30 20:09:25.816033 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.815882 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf"} err="failed to get container status \"17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf\": rpc error: code = NotFound desc = could not find container \"17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf\": container with ID starting with 17653e6b879f61aeeb2596eb28dafdd9c9dad5f9d86edc62d9191aa2985c9ebf not found: ID does not exist" Oct 30 20:09:25.816144 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.816037 6411 scope.go:122] "RemoveContainer" containerID="a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e" Oct 30 20:09:25.823528 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.823512 6411 scope.go:122] "RemoveContainer" containerID="a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e" Oct 30 20:09:25.823806 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:25.823787 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e\": container with ID starting with a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e not found: ID does not exist" containerID="a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e" Oct 30 20:09:25.823890 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.823815 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e"} err="failed to get container status \"a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e\": rpc error: code = NotFound desc = could not find container \"a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e\": container with ID starting with a6fea5cfba4938bbef576a8236c260299525a42db7ea000605ca3185e2a99b3e not found: ID does not exist" Oct 30 20:09:25.851676 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.851655 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt"] Oct 30 20:09:25.864595 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.864569 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-cm5lt"] Oct 30 20:09:25.915352 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.915333 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr"] Oct 30 20:09:25.925961 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.925939 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-jv9zr"] Oct 30 20:09:25.964431 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.964403 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl"] Oct 30 20:09:25.971336 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:25.971315 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-xksnl"] Oct 30 20:09:26.013750 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:26.013684 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9"] Oct 30 20:09:26.020716 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:26.020692 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-51dd62-openshift-default-7954bd8797-nzfg9"] Oct 30 20:09:26.397824 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:26.397790 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:27.482130 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.482096 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:27.713777 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.713742 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:27.731963 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.731936 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" path="/var/lib/kubelet/pods/546bae1b-d60e-47b5-8414-6fd7a0d66a5e/volumes" Oct 30 20:09:27.732415 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.732378 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" path="/var/lib/kubelet/pods/9733f9d7-b928-4bd7-be04-9ac327153c27/volumes" Oct 30 20:09:27.732768 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.732756 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" path="/var/lib/kubelet/pods/a210c8b6-cbb5-4ed7-b271-49e4912d42e6/volumes" Oct 30 20:09:27.733449 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.733435 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" path="/var/lib/kubelet/pods/d0964117-5aaf-4b8d-b0b1-ffbfc4496522/volumes" Oct 30 20:09:27.898032 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.897999 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr"] Oct 30 20:09:27.898278 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.898248 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" containerID="cri-o://74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1" gracePeriod=30 Oct 30 20:09:27.914663 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.914634 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h"] Oct 30 20:09:27.914880 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.914855 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" containerID="cri-o://a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02" gracePeriod=30 Oct 30 20:09:27.927210 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.927179 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8"] Oct 30 20:09:27.927428 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.927404 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" containerID="cri-o://a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4" gracePeriod=30 Oct 30 20:09:27.950351 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:27.950315 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:32.165211 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:32.165171 6411 patch_prober.go:29] interesting pod/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 20:09:32.165658 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:32.165267 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 20:09:32.915874 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:32.915839 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:33.110238 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.106453 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:33.281062 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.280755 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:09:33.359930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.344342 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:09:33.359930 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.346427 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:09:33.459527 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.459471 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-credential-socket\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.471132 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471051 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-podinfo\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471352 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istiod-ca-cert\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471392 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-data\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471425 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-token\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-token\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471464 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-kube-api-access-4tbz9\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-kube-api-access-4tbz9\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471496 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-credential-socket\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471533 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-credential-socket\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471566 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9738fae3-4f39-49cd-8368-02e1ea0134e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738fae3-4f39-49cd-8368-02e1ea0134e9-istiod-ca-cert\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471616 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-socket\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471664 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-envoy\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471695 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-kube-api-access-jbrqt\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-kube-api-access-jbrqt\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471726 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-envoy\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471783 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-token\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-token\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471823 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istiod-ca-cert\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471852 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-token\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-token\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471932 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-certs\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.471988 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-socket\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472025 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-certs\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472066 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-kube-api-access-fmbsp\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-kube-api-access-fmbsp\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472112 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-certs\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472163 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-envoy\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472205 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-socket\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472242 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-data\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472290 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-podinfo\") pod \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\" (UID: \"0e3323b5-cd7c-448d-a483-bd2052af9bdb\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472327 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-data\") pod \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\" (UID: \"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1\") " Oct 30 20:09:33.473134 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472373 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-podinfo\") pod \"9738fae3-4f39-49cd-8368-02e1ea0134e9\" (UID: \"9738fae3-4f39-49cd-8368-02e1ea0134e9\") " Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.472048 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-credential-socket" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.473014 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-credential-socket" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.473255 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-certs" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.473460 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-credential-socket" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.473720 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9738fae3-4f39-49cd-8368-02e1ea0134e9-istiod-ca-cert" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.475068 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-data" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.475380 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-certs" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.475740 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-certs" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.475957 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-data" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.476227 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istiod-ca-cert" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.476286 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-podinfo" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.476580 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-socket" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.476662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.476358 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-socket" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.477743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.477579 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-data" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.504923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.477868 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istiod-ca-cert" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:09:33.539489 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.512596 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-socket" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.565927 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.565848 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-kube-api-access-jbrqt" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "kube-api-access-jbrqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.575852 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.575886 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.575925 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.575940 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.575954 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.575968 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.575982 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.575997 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738fae3-4f39-49cd-8368-02e1ea0134e9-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576012 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576026 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbrqt\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-kube-api-access-jbrqt\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576042 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576055 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576069 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576082 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576096 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576109 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.576920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.576122 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.578123 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.578081 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-token" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:33.584953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.582159 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-envoy" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.585189 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.585152 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-kube-api-access-4tbz9" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "kube-api-access-4tbz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:33.585383 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.585346 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-token" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:33.586605 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.586548 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-kube-api-access-fmbsp" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "kube-api-access-fmbsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:33.586852 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.586815 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-token" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:09:33.587079 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.587041 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-envoy" pod "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" (UID: "b76a7e4b-02f3-448e-b296-8ee0ef7d90a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.610839 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.610794 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-podinfo" pod "9738fae3-4f39-49cd-8368-02e1ea0134e9" (UID: "9738fae3-4f39-49cd-8368-02e1ea0134e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:09:33.619076 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.619035 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-envoy" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:09:33.619315 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.619282 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-podinfo" pod "0e3323b5-cd7c-448d-a483-bd2052af9bdb" (UID: "0e3323b5-cd7c-448d-a483-bd2052af9bdb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677101 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmbsp\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-kube-api-access-fmbsp\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677137 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677154 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677167 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677181 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677194 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tbz9\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-kube-api-access-4tbz9\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677208 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677220 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677232 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e3323b5-cd7c-448d-a483-bd2052af9bdb-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.677273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.677244 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738fae3-4f39-49cd-8368-02e1ea0134e9-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.844449 6411 generic.go:356] "Generic (PLEG): container finished" podID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerID="a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4" exitCode=0 Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.844561 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" event={"ID":"0e3323b5-cd7c-448d-a483-bd2052af9bdb","Type":"ContainerDied","Data":"a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4"} Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.844615 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" event={"ID":"0e3323b5-cd7c-448d-a483-bd2052af9bdb","Type":"ContainerDied","Data":"a556782616dc30a397936874605a86214fe91b7fadc31f522d579a86db82aa87"} Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.844645 6411 scope.go:122] "RemoveContainer" containerID="a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4" Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.844785 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8" Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.851500 6411 generic.go:356] "Generic (PLEG): container finished" podID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerID="74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1" exitCode=0 Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.851601 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" event={"ID":"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1","Type":"ContainerDied","Data":"74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1"} Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.851642 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" event={"ID":"b76a7e4b-02f3-448e-b296-8ee0ef7d90a1","Type":"ContainerDied","Data":"4abfa7d64f4c79194d5995f84c1b1f91596a514e5071ad73d9d0b7a55cc0f9e4"} Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.851725 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr" Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.858375 6411 generic.go:356] "Generic (PLEG): container finished" podID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerID="a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02" exitCode=0 Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.858408 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" event={"ID":"9738fae3-4f39-49cd-8368-02e1ea0134e9","Type":"ContainerDied","Data":"a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02"} Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.858447 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" event={"ID":"9738fae3-4f39-49cd-8368-02e1ea0134e9","Type":"ContainerDied","Data":"b2f1b90375aec69886cf29eba504f699b196c7d6cd0dce500848ee8c8b765bd8"} Oct 30 20:09:33.863992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.858513 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h" Oct 30 20:09:33.975018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.965026 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8"] Oct 30 20:09:33.975018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.965409 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-cmwz8"] Oct 30 20:09:33.975018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.970045 6411 scope.go:122] "RemoveContainer" containerID="a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4" Oct 30 20:09:33.975646 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:33.975510 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4\": container with ID starting with a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4 not found: ID does not exist" containerID="a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4" Oct 30 20:09:33.975646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.975548 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4"} err="failed to get container status \"a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4\": rpc error: code = NotFound desc = could not find container \"a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4\": container with ID starting with a54815979fe7c37b08a22bbc1c4b69e17635c5a8c5d521efe4d7d41d70d4e0f4 not found: ID does not exist" Oct 30 20:09:33.975646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.975575 6411 scope.go:122] "RemoveContainer" containerID="74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1" Oct 30 20:09:33.977702 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:33.977659 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h"] Oct 30 20:09:34.005042 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.003533 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-45h7h"] Oct 30 20:09:34.051941 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.046743 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr"] Oct 30 20:09:34.064973 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.064772 6411 scope.go:122] "RemoveContainer" containerID="74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1" Oct 30 20:09:34.064973 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.064954 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-efbac3-openshift-default-844cb4689c-4ltxr"] Oct 30 20:09:34.065261 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:34.065235 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1\": container with ID starting with 74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1 not found: ID does not exist" containerID="74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1" Oct 30 20:09:34.065343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.065271 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1"} err="failed to get container status \"74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1\": rpc error: code = NotFound desc = could not find container \"74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1\": container with ID starting with 74e916d2440a308d247a1742b0977430b0f00774d3d539b330990f4351a21bd1 not found: ID does not exist" Oct 30 20:09:34.065343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.065298 6411 scope.go:122] "RemoveContainer" containerID="a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02" Oct 30 20:09:34.073929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.073886 6411 scope.go:122] "RemoveContainer" containerID="a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02" Oct 30 20:09:34.074156 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:34.074141 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02\": container with ID starting with a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02 not found: ID does not exist" containerID="a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02" Oct 30 20:09:34.074209 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.074160 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02"} err="failed to get container status \"a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02\": rpc error: code = NotFound desc = could not find container \"a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02\": container with ID starting with a5603b25c432776d8b7023981d09d5ba9c4579871197bb3a0d38b375957cad02 not found: ID does not exist" Oct 30 20:09:34.505924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.497001 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:09:34.505924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.497073 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:09:34.557553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.557025 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-smpxp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:09:34.557553 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:34.557132 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podUID="f2139ca9-f068-4350-8d67-00ce2a968911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:09:35.127878 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:09:35.127840 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:09:35.732175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:35.732137 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" path="/var/lib/kubelet/pods/0e3323b5-cd7c-448d-a483-bd2052af9bdb/volumes" Oct 30 20:09:35.732771 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:35.732749 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" path="/var/lib/kubelet/pods/9738fae3-4f39-49cd-8368-02e1ea0134e9/volumes" Oct 30 20:09:35.733226 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:35.733206 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" path="/var/lib/kubelet/pods/b76a7e4b-02f3-448e-b296-8ee0ef7d90a1/volumes" Oct 30 20:09:36.195478 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:36.195437 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:42.555238 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:42.555202 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:46.739425 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:46.739389 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:53.142468 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:53.142435 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:09:53.350780 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:09:53.350744 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:10:05.096795 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:10:05.096747 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:10:08.790272 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:10:08.790228 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:10:11.096107 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:10:11.096075 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:10:11.878583 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:10:11.878552 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:10:12.238196 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:10:12.238154 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:10:19.166238 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:10:19.166201 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:10:31.293368 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:10:31.293333 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:10:31.500878 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:10:31.500841 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:10:35.091386 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:10:35.091350 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:10:56.616225 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:10:56.616186 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:11:04.477083 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:11:04.477040 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:11:04.479252 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:11:04.477112 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:11:05.093271 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:11:05.093223 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:11:11.993979 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:11:11.993941 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:11:12.367716 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:11:12.367636 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:11:35.164743 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:11:35.164678 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:12:05.092669 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:12:05.092629 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:12:12.175579 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:12:12.175489 6411 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" limit=1024 Oct 30 20:12:35.092084 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:12:35.092044 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:13:05.091798 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:13:05.091764 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:13:35.091242 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:13:35.091197 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:13:47.431737 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.431700 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw"] Oct 30 20:13:47.432278 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.431968 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" podUID="ee40c7b0-1679-4983-a004-ddc225786c80" containerName="discovery" containerID="cri-o://167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6" gracePeriod=30 Oct 30 20:13:47.706443 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.706415 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:13:47.841090 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841057 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-kubeconfig\") pod \"ee40c7b0-1679-4983-a004-ddc225786c80\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " Oct 30 20:13:47.841251 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841106 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-istio-token\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-istio-token\") pod \"ee40c7b0-1679-4983-a004-ddc225786c80\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " Oct 30 20:13:47.841251 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841206 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-cacerts\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-cacerts\") pod \"ee40c7b0-1679-4983-a004-ddc225786c80\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " Oct 30 20:13:47.841251 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841239 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee40c7b0-1679-4983-a004-ddc225786c80-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee40c7b0-1679-4983-a004-ddc225786c80-local-certs\") pod \"ee40c7b0-1679-4983-a004-ddc225786c80\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " Oct 30 20:13:47.841412 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841273 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-ca-configmap\") pod \"ee40c7b0-1679-4983-a004-ddc225786c80\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " Oct 30 20:13:47.841412 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841306 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-dns-cert\") pod \"ee40c7b0-1679-4983-a004-ddc225786c80\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " Oct 30 20:13:47.841412 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841344 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-kube-api-access-8ll6t\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-kube-api-access-8ll6t\") pod \"ee40c7b0-1679-4983-a004-ddc225786c80\" (UID: \"ee40c7b0-1679-4983-a004-ddc225786c80\") " Oct 30 20:13:47.841649 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841617 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-ca-configmap" pod "ee40c7b0-1679-4983-a004-ddc225786c80" (UID: "ee40c7b0-1679-4983-a004-ddc225786c80"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:13:47.841852 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.841830 6411 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-ca-configmap\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:13:47.843662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.843628 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-istio-token" pod "ee40c7b0-1679-4983-a004-ddc225786c80" (UID: "ee40c7b0-1679-4983-a004-ddc225786c80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:13:47.844098 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.844070 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee40c7b0-1679-4983-a004-ddc225786c80-local-certs" pod "ee40c7b0-1679-4983-a004-ddc225786c80" (UID: "ee40c7b0-1679-4983-a004-ddc225786c80"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:13:47.844228 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.844208 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-kubeconfig" pod "ee40c7b0-1679-4983-a004-ddc225786c80" (UID: "ee40c7b0-1679-4983-a004-ddc225786c80"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:13:47.844289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.844241 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-dns-cert" pod "ee40c7b0-1679-4983-a004-ddc225786c80" (UID: "ee40c7b0-1679-4983-a004-ddc225786c80"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:13:47.844333 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.844298 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-cacerts" pod "ee40c7b0-1679-4983-a004-ddc225786c80" (UID: "ee40c7b0-1679-4983-a004-ddc225786c80"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:13:47.844415 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.844392 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-kube-api-access-8ll6t" pod "ee40c7b0-1679-4983-a004-ddc225786c80" (UID: "ee40c7b0-1679-4983-a004-ddc225786c80"). InnerVolumeSpecName "kube-api-access-8ll6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:13:47.942608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.942513 6411 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-cacerts\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:13:47.942608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.942549 6411 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee40c7b0-1679-4983-a004-ddc225786c80-local-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:13:47.942608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.942563 6411 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-csr-dns-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:13:47.942608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.942577 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ll6t\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-kube-api-access-8ll6t\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:13:47.942608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.942588 6411 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ee40c7b0-1679-4983-a004-ddc225786c80-istio-kubeconfig\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:13:47.942608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:47.942600 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee40c7b0-1679-4983-a004-ddc225786c80-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:13:48.112125 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.112092 6411 generic.go:356] "Generic (PLEG): container finished" podID="ee40c7b0-1679-4983-a004-ddc225786c80" containerID="167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6" exitCode=0 Oct 30 20:13:48.112329 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.112136 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" Oct 30 20:13:48.112329 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.112173 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" event={"ID":"ee40c7b0-1679-4983-a004-ddc225786c80","Type":"ContainerDied","Data":"167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6"} Oct 30 20:13:48.112329 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.112226 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw" event={"ID":"ee40c7b0-1679-4983-a004-ddc225786c80","Type":"ContainerDied","Data":"e8f724faa4f6a5be0b0c718eeaaf762959dfde6cd4936af51fde1b420a4d5fda"} Oct 30 20:13:48.112329 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.112246 6411 scope.go:122] "RemoveContainer" containerID="167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6" Oct 30 20:13:48.121399 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.121376 6411 scope.go:122] "RemoveContainer" containerID="167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6" Oct 30 20:13:48.121673 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:13:48.121655 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6\": container with ID starting with 167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6 not found: ID does not exist" containerID="167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6" Oct 30 20:13:48.121754 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.121681 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6"} err="failed to get container status \"167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6\": rpc error: code = NotFound desc = could not find container \"167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6\": container with ID starting with 167d7d45304bc0897279980bf59c9975a48a4ae1435aa4f83237fe26cf69d1b6 not found: ID does not exist" Oct 30 20:13:48.168619 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.168597 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw"] Oct 30 20:13:48.181716 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:48.181693 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nwmhw"] Oct 30 20:13:49.732315 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:13:49.732289 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee40c7b0-1679-4983-a004-ddc225786c80" path="/var/lib/kubelet/pods/ee40c7b0-1679-4983-a004-ddc225786c80/volumes" Oct 30 20:14:05.091605 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:14:05.091567 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:14:34.617711 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.617680 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2"] Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618082 6411 cpu_manager.go:414] "containerMap: removing container" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618096 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618106 6411 cpu_manager.go:414] "containerMap: removing container" podUID="aeddd424-8ad6-40dd-aaec-68195c43415e" containerName="payload-pre-processing" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618111 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aeddd424-8ad6-40dd-aaec-68195c43415e" containerName="payload-pre-processing" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618118 6411 cpu_manager.go:414] "containerMap: removing container" podUID="ee40c7b0-1679-4983-a004-ddc225786c80" containerName="discovery" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618123 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee40c7b0-1679-4983-a004-ddc225786c80" containerName="discovery" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618131 6411 cpu_manager.go:414] "containerMap: removing container" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618136 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618144 6411 cpu_manager.go:414] "containerMap: removing container" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618149 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618161 6411 cpu_manager.go:414] "containerMap: removing container" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618168 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618174 6411 cpu_manager.go:414] "containerMap: removing container" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618179 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618184 6411 cpu_manager.go:414] "containerMap: removing container" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618188 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618199 6411 cpu_manager.go:414] "containerMap: removing container" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618203 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618211 6411 cpu_manager.go:414] "containerMap: removing container" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618216 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618222 6411 cpu_manager.go:414] "containerMap: removing container" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618228 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618240 6411 cpu_manager.go:414] "containerMap: removing container" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618245 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" Oct 30 20:14:34.618242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618252 6411 cpu_manager.go:414] "containerMap: removing container" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618257 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618267 6411 cpu_manager.go:414] "containerMap: removing container" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618271 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618277 6411 cpu_manager.go:414] "containerMap: removing container" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618281 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618288 6411 cpu_manager.go:414] "containerMap: removing container" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618292 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618297 6411 cpu_manager.go:414] "containerMap: removing container" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618301 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618314 6411 cpu_manager.go:414] "containerMap: removing container" podUID="b0c2935f-803d-4343-94a7-af159d12e098" containerName="maas-api" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618319 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0c2935f-803d-4343-94a7-af159d12e098" containerName="maas-api" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618325 6411 cpu_manager.go:414] "containerMap: removing container" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618329 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618343 6411 cpu_manager.go:414] "containerMap: removing container" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618347 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618415 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0c2935f-803d-4343-94a7-af159d12e098" containerName="maas-api" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618421 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="708fc137-602b-42fc-9572-7aacdbffa8cb" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618429 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="860bfb55-555b-4198-b9ae-23712a7453e1" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618433 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618438 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="aeddd424-8ad6-40dd-aaec-68195c43415e" containerName="payload-pre-processing" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618443 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee40c7b0-1679-4983-a004-ddc225786c80" containerName="discovery" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618447 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="b76a7e4b-02f3-448e-b296-8ee0ef7d90a1" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618451 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="a210c8b6-cbb5-4ed7-b271-49e4912d42e6" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618456 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618464 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0964117-5aaf-4b8d-b0b1-ffbfc4496522" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618471 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b7d73fb-1fb8-4969-a783-f242346001f2" containerName="main" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618477 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e3323b5-cd7c-448d-a483-bd2052af9bdb" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618482 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="20a8ade9-4b09-47c2-b7f5-ec0d6ad8f5be" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618487 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="85e7c420-1e78-42cf-abc6-25366f1e9387" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618491 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="546bae1b-d60e-47b5-8414-6fd7a0d66a5e" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618497 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="b71651b4-40f9-4900-aa6d-84328edffa6d" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618503 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6a26c55-a188-4bd8-ba15-60bb3a5deb2c" containerName="main" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618508 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="9733f9d7-b928-4bd7-be04-9ac327153c27" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618513 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="9738fae3-4f39-49cd-8368-02e1ea0134e9" containerName="istio-proxy" Oct 30 20:14:34.619019 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.618517 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7401270-59d3-4f59-af0c-c7943dde243b" containerName="istio-proxy" Oct 30 20:14:34.621987 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.621969 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.642102 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.642080 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-a989da-openshift-default-dockercfg-v7whm\"" Oct 30 20:14:34.653379 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.653344 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2"] Oct 30 20:14:34.795043 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795009 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-envoy\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.795221 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795063 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv9nj\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-kube-api-access-xv9nj\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.795221 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795153 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-socket\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.795221 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795200 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istiod-ca-cert\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.795377 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795272 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-credential-socket\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.795377 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795307 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-token\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.795455 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795389 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-podinfo\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.795455 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795414 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-data\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.795455 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.795450 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-certs\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.896718 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.896687 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-certs\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.896953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.896738 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-envoy\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.896953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.896777 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv9nj\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-kube-api-access-xv9nj\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.896953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.896806 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-socket\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.896953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.896845 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istiod-ca-cert\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.896997 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-credential-socket\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.897071 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-token\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.897147 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-podinfo\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.897169 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-certs\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.897181 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-data\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897480 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.897296 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-socket\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.897509 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-credential-socket\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897541 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.897529 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-data\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.897649 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.897568 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istiod-ca-cert\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.899463 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.899439 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-envoy\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.899803 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.899784 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-podinfo\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.922201 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.922179 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-token\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.922660 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.922634 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv9nj\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-kube-api-access-xv9nj\") pod \"e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:34.937419 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:34.937386 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:35.092211 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:14:35.092163 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:14:35.255947 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:35.255921 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2"] Oct 30 20:14:35.258279 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:14:35.258253 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca522e5c_d8e7_4bba_b105_4a4794d37db6.slice/crio-26a4be8362dee52439b802af35a445f2fb12d1d86a4119ca149cf49e1216d4bc WatchSource:0}: Error finding container 26a4be8362dee52439b802af35a445f2fb12d1d86a4119ca149cf49e1216d4bc: Status 404 returned error can't find the container with id 26a4be8362dee52439b802af35a445f2fb12d1d86a4119ca149cf49e1216d4bc Oct 30 20:14:35.260635 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:35.260602 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:14:35.260745 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:35.260672 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:14:35.260745 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:35.260726 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:14:35.337587 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:35.337558 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" event={"ID":"ca522e5c-d8e7-4bba-b105-4a4794d37db6","Type":"ContainerStarted","Data":"493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800"} Oct 30 20:14:35.337740 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:35.337608 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" event={"ID":"ca522e5c-d8e7-4bba-b105-4a4794d37db6","Type":"ContainerStarted","Data":"26a4be8362dee52439b802af35a445f2fb12d1d86a4119ca149cf49e1216d4bc"} Oct 30 20:14:35.412439 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:35.412393 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" podStartSLOduration=1.412379274 podStartE2EDuration="1.412379274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:14:35.410394653 +0000 UTC m=+1591.810566660" watchObservedRunningTime="2026-10-30 20:14:35.412379274 +0000 UTC m=+1591.812551277" Oct 30 20:14:35.938821 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:35.938789 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:36.943238 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:36.943208 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:37.346454 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:37.346373 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:37.347336 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:37.347317 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:39.585098 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.585065 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k"] Oct 30 20:14:39.588781 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.588761 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.611438 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.611405 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-a989da-serving-cert\"" Oct 30 20:14:39.624471 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.624442 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k"] Oct 30 20:14:39.640984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.640943 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.641158 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.641037 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c040bbce-dcc8-4c36-87b4-b60e508c0031-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.641158 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.641100 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds7ql\" (UniqueName: \"kubernetes.io/projected/c040bbce-dcc8-4c36-87b4-b60e508c0031-kube-api-access-ds7ql\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.641293 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.641171 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.741662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.741635 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.741934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.741691 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.741934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.741817 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c040bbce-dcc8-4c36-87b4-b60e508c0031-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.741934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.741868 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds7ql\" (UniqueName: \"kubernetes.io/projected/c040bbce-dcc8-4c36-87b4-b60e508c0031-kube-api-access-ds7ql\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.742817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.742788 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c040bbce-dcc8-4c36-87b4-b60e508c0031-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.744157 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.744138 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.744277 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.744257 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.768232 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.768198 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds7ql\" (UniqueName: \"kubernetes.io/projected/c040bbce-dcc8-4c36-87b4-b60e508c0031-kube-api-access-ds7ql\") pod \"maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:39.905287 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:39.905238 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:40.219206 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:40.219184 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k"] Oct 30 20:14:40.221160 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:14:40.221133 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc040bbce_dcc8_4c36_87b4_b60e508c0031.slice/crio-adbbbb08f539cfd64e6ab88649b376dca229adadcbf8c6c50989fe3c03c3b4ab WatchSource:0}: Error finding container adbbbb08f539cfd64e6ab88649b376dca229adadcbf8c6c50989fe3c03c3b4ab: Status 404 returned error can't find the container with id adbbbb08f539cfd64e6ab88649b376dca229adadcbf8c6c50989fe3c03c3b4ab Oct 30 20:14:40.222634 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:40.222618 6411 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 20:14:40.362029 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:40.361994 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" event={"ID":"c040bbce-dcc8-4c36-87b4-b60e508c0031","Type":"ContainerStarted","Data":"adbbbb08f539cfd64e6ab88649b376dca229adadcbf8c6c50989fe3c03c3b4ab"} Oct 30 20:14:41.368426 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:41.368393 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" event={"ID":"c040bbce-dcc8-4c36-87b4-b60e508c0031","Type":"ContainerStarted","Data":"75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda"} Oct 30 20:14:41.368863 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:41.368454 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:41.417216 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:41.417118 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" podStartSLOduration=2.052500945 podStartE2EDuration="2.417097617s" totalImagesPullingTime="364.596672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:14:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:14:41.414164468 +0000 UTC m=+1597.814336476" watchObservedRunningTime="2026-10-30 20:14:41.417097617 +0000 UTC m=+1597.817269627" Oct 30 20:14:47.406922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:47.391199 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:14:52.471355 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:52.470957 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": read tcp 10.133.0.2:33460->10.133.0.80:15021: read: connection reset by peer" start-of-body= Oct 30 20:14:52.471355 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:52.471109 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": read tcp 10.133.0.2:33460->10.133.0.80:15021: read: connection reset by peer" Oct 30 20:14:53.445933 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:53.445879 6411 generic.go:356] "Generic (PLEG): container finished" podID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerID="493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800" exitCode=0 Oct 30 20:14:53.446113 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:53.445955 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" event={"ID":"ca522e5c-d8e7-4bba-b105-4a4794d37db6","Type":"ContainerDied","Data":"493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800"} Oct 30 20:14:53.446380 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:53.446365 6411 scope.go:122] "RemoveContainer" containerID="493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800" Oct 30 20:14:53.447627 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:53.447598 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:14:53.447694 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:53.447667 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:14:53.447734 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:53.447699 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:14:54.452854 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:54.452817 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" event={"ID":"ca522e5c-d8e7-4bba-b105-4a4794d37db6","Type":"ContainerStarted","Data":"e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63"} Oct 30 20:14:54.938790 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:54.938756 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:14:54.939005 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:54.938979 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Oct 30 20:14:54.939058 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:54.939030 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Oct 30 20:14:55.938063 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:55.938031 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Oct 30 20:14:55.938433 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:55.938091 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Oct 30 20:14:56.938811 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:56.938779 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Oct 30 20:14:56.939225 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:56.938831 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Oct 30 20:14:57.938881 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:57.938838 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Oct 30 20:14:57.939317 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:57.938927 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Oct 30 20:14:58.960920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:58.956340 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Oct 30 20:14:58.960920 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:14:58.956409 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Oct 30 20:15:00.125792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:00.125757 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:15:00.126309 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:00.126209 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:15:00.127427 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:00.127405 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:15:05.092245 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:15:05.092207 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:15:35.094041 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:15:35.093961 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:15:40.846057 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:40.846018 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k"] Oct 30 20:15:40.846534 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:40.846279 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" podUID="c040bbce-dcc8-4c36-87b4-b60e508c0031" containerName="maas-api" containerID="cri-o://75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda" gracePeriod=30 Oct 30 20:15:41.314509 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.314481 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:15:41.355477 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.355446 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-metrics-certs\") pod \"c040bbce-dcc8-4c36-87b4-b60e508c0031\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " Oct 30 20:15:41.355664 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.355544 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c040bbce-dcc8-4c36-87b4-b60e508c0031-kube-api-access-ds7ql\" (UniqueName: \"kubernetes.io/projected/c040bbce-dcc8-4c36-87b4-b60e508c0031-kube-api-access-ds7ql\") pod \"c040bbce-dcc8-4c36-87b4-b60e508c0031\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " Oct 30 20:15:41.355664 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.355619 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-maas-api-tls\") pod \"c040bbce-dcc8-4c36-87b4-b60e508c0031\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " Oct 30 20:15:41.355664 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.355659 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c040bbce-dcc8-4c36-87b4-b60e508c0031-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c040bbce-dcc8-4c36-87b4-b60e508c0031-openshift-service-ca\") pod \"c040bbce-dcc8-4c36-87b4-b60e508c0031\" (UID: \"c040bbce-dcc8-4c36-87b4-b60e508c0031\") " Oct 30 20:15:41.356178 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.356137 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c040bbce-dcc8-4c36-87b4-b60e508c0031-openshift-service-ca" pod "c040bbce-dcc8-4c36-87b4-b60e508c0031" (UID: "c040bbce-dcc8-4c36-87b4-b60e508c0031"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:15:41.357815 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.357782 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-metrics-certs" pod "c040bbce-dcc8-4c36-87b4-b60e508c0031" (UID: "c040bbce-dcc8-4c36-87b4-b60e508c0031"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:15:41.358414 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.358392 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-maas-api-tls" pod "c040bbce-dcc8-4c36-87b4-b60e508c0031" (UID: "c040bbce-dcc8-4c36-87b4-b60e508c0031"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:15:41.358540 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.358427 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c040bbce-dcc8-4c36-87b4-b60e508c0031-kube-api-access-ds7ql" pod "c040bbce-dcc8-4c36-87b4-b60e508c0031" (UID: "c040bbce-dcc8-4c36-87b4-b60e508c0031"). InnerVolumeSpecName "kube-api-access-ds7ql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:15:41.457229 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.457137 6411 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c040bbce-dcc8-4c36-87b4-b60e508c0031-openshift-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:41.457229 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.457168 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:41.457229 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.457178 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds7ql\" (UniqueName: \"kubernetes.io/projected/c040bbce-dcc8-4c36-87b4-b60e508c0031-kube-api-access-ds7ql\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:41.457229 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.457186 6411 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c040bbce-dcc8-4c36-87b4-b60e508c0031-maas-api-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:41.670401 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.670363 6411 generic.go:356] "Generic (PLEG): container finished" podID="c040bbce-dcc8-4c36-87b4-b60e508c0031" containerID="75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda" exitCode=0 Oct 30 20:15:41.670583 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.670419 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" Oct 30 20:15:41.670583 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.670437 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" event={"ID":"c040bbce-dcc8-4c36-87b4-b60e508c0031","Type":"ContainerDied","Data":"75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda"} Oct 30 20:15:41.670583 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.670489 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k" event={"ID":"c040bbce-dcc8-4c36-87b4-b60e508c0031","Type":"ContainerDied","Data":"adbbbb08f539cfd64e6ab88649b376dca229adadcbf8c6c50989fe3c03c3b4ab"} Oct 30 20:15:41.670583 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.670510 6411 scope.go:122] "RemoveContainer" containerID="75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda" Oct 30 20:15:41.679693 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.679667 6411 scope.go:122] "RemoveContainer" containerID="75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda" Oct 30 20:15:41.680003 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:15:41.679984 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda\": container with ID starting with 75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda not found: ID does not exist" containerID="75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda" Oct 30 20:15:41.680070 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.680010 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda"} err="failed to get container status \"75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda\": rpc error: code = NotFound desc = could not find container \"75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda\": container with ID starting with 75ee3d98a31a05b98e762c04a1674f987a57da402cd5f18d879afd7cba0eedda not found: ID does not exist" Oct 30 20:15:41.732585 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.732561 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k"] Oct 30 20:15:41.742830 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:41.742807 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a989da-6fb47c79c5-gjv5k"] Oct 30 20:15:43.733608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:43.733567 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c040bbce-dcc8-4c36-87b4-b60e508c0031" path="/var/lib/kubelet/pods/c040bbce-dcc8-4c36-87b4-b60e508c0031/volumes" Oct 30 20:15:50.134658 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:50.134617 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2"] Oct 30 20:15:50.135260 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:50.134878 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" containerID="cri-o://e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63" gracePeriod=30 Oct 30 20:15:55.386045 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.386022 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:15:55.501962 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.501863 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-envoy\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.501962 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.501921 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-kube-api-access-xv9nj\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-kube-api-access-xv9nj\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.501962 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.501956 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-podinfo\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.502181 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502139 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-certs\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.502243 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502217 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-socket\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.502295 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502243 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-token\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-token\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.502359 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502335 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-credential-socket\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.502472 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502351 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-certs" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:15:55.502472 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502391 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-data\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.502472 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502443 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istiod-ca-cert\") pod \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\" (UID: \"ca522e5c-d8e7-4bba-b105-4a4794d37db6\") " Oct 30 20:15:55.502792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502478 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-socket" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:15:55.502792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502661 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-data" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:15:55.502792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502699 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-credential-socket" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:15:55.502792 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502772 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.503046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502816 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.503046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502833 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.503046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502848 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.503046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.502948 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istiod-ca-cert" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:15:55.504186 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.504167 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-envoy" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:15:55.504548 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.504535 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-token" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:15:55.504688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.504660 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-podinfo" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:15:55.504688 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.504671 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-kube-api-access-xv9nj" pod "ca522e5c-d8e7-4bba-b105-4a4794d37db6" (UID: "ca522e5c-d8e7-4bba-b105-4a4794d37db6"). InnerVolumeSpecName "kube-api-access-xv9nj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:15:55.604290 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.604261 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.604290 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.604285 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.604290 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.604294 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv9nj\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-kube-api-access-xv9nj\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.604519 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.604306 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.604519 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.604314 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca522e5c-d8e7-4bba-b105-4a4794d37db6-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:15:55.734807 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.734773 6411 generic.go:356] "Generic (PLEG): container finished" podID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerID="e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63" exitCode=0 Oct 30 20:15:55.735013 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.734845 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" event={"ID":"ca522e5c-d8e7-4bba-b105-4a4794d37db6","Type":"ContainerDied","Data":"e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63"} Oct 30 20:15:55.735013 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.734854 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" Oct 30 20:15:55.735013 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.734919 6411 scope.go:122] "RemoveContainer" containerID="e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63" Oct 30 20:15:55.735013 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.734906 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2" event={"ID":"ca522e5c-d8e7-4bba-b105-4a4794d37db6","Type":"ContainerDied","Data":"26a4be8362dee52439b802af35a445f2fb12d1d86a4119ca149cf49e1216d4bc"} Oct 30 20:15:55.744177 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.744161 6411 scope.go:122] "RemoveContainer" containerID="493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800" Oct 30 20:15:55.752145 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.752129 6411 scope.go:122] "RemoveContainer" containerID="e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63" Oct 30 20:15:55.752426 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:15:55.752409 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63\": container with ID starting with e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63 not found: ID does not exist" containerID="e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63" Oct 30 20:15:55.752476 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.752437 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63"} err="failed to get container status \"e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63\": rpc error: code = NotFound desc = could not find container \"e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63\": container with ID starting with e800186e51d8c418366f52edf95970e43e96ff48a188fa277a7096a6b5e07b63 not found: ID does not exist" Oct 30 20:15:55.752476 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.752453 6411 scope.go:122] "RemoveContainer" containerID="493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800" Oct 30 20:15:55.752731 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:15:55.752687 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800\": container with ID starting with 493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800 not found: ID does not exist" containerID="493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800" Oct 30 20:15:55.752731 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.752710 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800"} err="failed to get container status \"493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800\": rpc error: code = NotFound desc = could not find container \"493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800\": container with ID starting with 493afae6fc8314c8c35fc45fe6da4ca6853f3fc8c57025f2ca4d7afcda6ae800 not found: ID does not exist" Oct 30 20:15:55.806616 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.806589 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2"] Oct 30 20:15:55.821242 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:55.821220 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a989da-openshift-default-849677bc5cf7wl2"] Oct 30 20:15:57.732097 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:15:57.732067 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" path="/var/lib/kubelet/pods/ca522e5c-d8e7-4bba-b105-4a4794d37db6/volumes" Oct 30 20:16:05.157185 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:16:05.157121 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:16:15.797220 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:15.797188 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h"] Oct 30 20:16:15.797796 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:15.797458 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" podUID="88050ac8-8997-4c0c-8f10-4e3d3452a740" containerName="manager" containerID="cri-o://9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177" gracePeriod=10 Oct 30 20:16:16.060729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.060678 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 20:16:16.221987 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.221958 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88050ac8-8997-4c0c-8f10-4e3d3452a740-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88050ac8-8997-4c0c-8f10-4e3d3452a740-extensions-socket-volume\") pod \"88050ac8-8997-4c0c-8f10-4e3d3452a740\" (UID: \"88050ac8-8997-4c0c-8f10-4e3d3452a740\") " Oct 30 20:16:16.222161 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.221998 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88050ac8-8997-4c0c-8f10-4e3d3452a740-kube-api-access-lg6nz\" (UniqueName: \"kubernetes.io/projected/88050ac8-8997-4c0c-8f10-4e3d3452a740-kube-api-access-lg6nz\") pod \"88050ac8-8997-4c0c-8f10-4e3d3452a740\" (UID: \"88050ac8-8997-4c0c-8f10-4e3d3452a740\") " Oct 30 20:16:16.222308 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.222287 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88050ac8-8997-4c0c-8f10-4e3d3452a740-extensions-socket-volume" pod "88050ac8-8997-4c0c-8f10-4e3d3452a740" (UID: "88050ac8-8997-4c0c-8f10-4e3d3452a740"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:16:16.224082 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.224059 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88050ac8-8997-4c0c-8f10-4e3d3452a740-kube-api-access-lg6nz" pod "88050ac8-8997-4c0c-8f10-4e3d3452a740" (UID: "88050ac8-8997-4c0c-8f10-4e3d3452a740"). InnerVolumeSpecName "kube-api-access-lg6nz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:16:16.322800 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.322735 6411 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88050ac8-8997-4c0c-8f10-4e3d3452a740-extensions-socket-volume\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:16:16.322800 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.322760 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg6nz\" (UniqueName: \"kubernetes.io/projected/88050ac8-8997-4c0c-8f10-4e3d3452a740-kube-api-access-lg6nz\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:16:16.826860 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.826821 6411 generic.go:356] "Generic (PLEG): container finished" podID="88050ac8-8997-4c0c-8f10-4e3d3452a740" containerID="9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177" exitCode=0 Oct 30 20:16:16.827306 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.826857 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" event={"ID":"88050ac8-8997-4c0c-8f10-4e3d3452a740","Type":"ContainerDied","Data":"9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177"} Oct 30 20:16:16.827306 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.826868 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" Oct 30 20:16:16.827306 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.826940 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h" event={"ID":"88050ac8-8997-4c0c-8f10-4e3d3452a740","Type":"ContainerDied","Data":"b37181cf46d55a77659616d552fc05dcd2eedcb9d43a4422301068ab8f59157c"} Oct 30 20:16:16.827306 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.826966 6411 scope.go:122] "RemoveContainer" containerID="9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177" Oct 30 20:16:16.840337 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.840321 6411 scope.go:122] "RemoveContainer" containerID="9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177" Oct 30 20:16:16.840631 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:16:16.840611 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177\": container with ID starting with 9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177 not found: ID does not exist" containerID="9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177" Oct 30 20:16:16.840682 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.840639 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177"} err="failed to get container status \"9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177\": rpc error: code = NotFound desc = could not find container \"9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177\": container with ID starting with 9d9e886951badbc8173e6775e267d9cd259c5ae3317337c43f017004ea558177 not found: ID does not exist" Oct 30 20:16:16.894821 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.894797 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h"] Oct 30 20:16:16.907576 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:16.907553 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vn25h"] Oct 30 20:16:17.732216 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:16:17.732188 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88050ac8-8997-4c0c-8f10-4e3d3452a740" path="/var/lib/kubelet/pods/88050ac8-8997-4c0c-8f10-4e3d3452a740/volumes" Oct 30 20:16:35.090600 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:16:35.090554 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:17:05.091383 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:17:05.091346 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:17:18.956746 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.956708 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46"] Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957132 6411 cpu_manager.go:414] "containerMap: removing container" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957145 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957161 6411 cpu_manager.go:414] "containerMap: removing container" podUID="88050ac8-8997-4c0c-8f10-4e3d3452a740" containerName="manager" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957166 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88050ac8-8997-4c0c-8f10-4e3d3452a740" containerName="manager" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957178 6411 cpu_manager.go:414] "containerMap: removing container" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957183 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957199 6411 cpu_manager.go:414] "containerMap: removing container" podUID="c040bbce-dcc8-4c36-87b4-b60e508c0031" containerName="maas-api" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957204 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c040bbce-dcc8-4c36-87b4-b60e508c0031" containerName="maas-api" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957257 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957264 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="88050ac8-8997-4c0c-8f10-4e3d3452a740" containerName="manager" Oct 30 20:17:18.957289 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.957274 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="c040bbce-dcc8-4c36-87b4-b60e508c0031" containerName="maas-api" Oct 30 20:17:18.961343 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.961326 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:18.990731 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.990704 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5pgqn\"" Oct 30 20:17:18.992018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:18.992000 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46"] Oct 30 20:17:19.056572 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:19.056545 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a69ad986-9fba-4a09-9d5f-65236575a370-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-wxz46\" (UID: \"a69ad986-9fba-4a09-9d5f-65236575a370\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:19.056714 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:19.056603 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b965n\" (UniqueName: \"kubernetes.io/projected/a69ad986-9fba-4a09-9d5f-65236575a370-kube-api-access-b965n\") pod \"kuadrant-operator-controller-manager-57bb9bfff-wxz46\" (UID: \"a69ad986-9fba-4a09-9d5f-65236575a370\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:19.157419 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:19.157385 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a69ad986-9fba-4a09-9d5f-65236575a370-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-wxz46\" (UID: \"a69ad986-9fba-4a09-9d5f-65236575a370\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:19.157549 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:19.157449 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b965n\" (UniqueName: \"kubernetes.io/projected/a69ad986-9fba-4a09-9d5f-65236575a370-kube-api-access-b965n\") pod \"kuadrant-operator-controller-manager-57bb9bfff-wxz46\" (UID: \"a69ad986-9fba-4a09-9d5f-65236575a370\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:19.157767 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:19.157748 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a69ad986-9fba-4a09-9d5f-65236575a370-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-wxz46\" (UID: \"a69ad986-9fba-4a09-9d5f-65236575a370\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:19.183614 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:19.183586 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b965n\" (UniqueName: \"kubernetes.io/projected/a69ad986-9fba-4a09-9d5f-65236575a370-kube-api-access-b965n\") pod \"kuadrant-operator-controller-manager-57bb9bfff-wxz46\" (UID: \"a69ad986-9fba-4a09-9d5f-65236575a370\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:19.275048 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:19.274987 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:19.600485 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:19.600460 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46"] Oct 30 20:17:19.602587 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:17:19.602557 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69ad986_9fba_4a09_9d5f_65236575a370.slice/crio-b8a5a2969b48376d58611b2de5725d1bc39be0a29882a1d047e4f4a1da621656 WatchSource:0}: Error finding container b8a5a2969b48376d58611b2de5725d1bc39be0a29882a1d047e4f4a1da621656: Status 404 returned error can't find the container with id b8a5a2969b48376d58611b2de5725d1bc39be0a29882a1d047e4f4a1da621656 Oct 30 20:17:20.096502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:20.096455 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" event={"ID":"a69ad986-9fba-4a09-9d5f-65236575a370","Type":"ContainerStarted","Data":"03a1d8fe284f45a9536db1381a41a326b796e7f0109e5cbf774ae43feba2d826"} Oct 30 20:17:20.097117 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:20.097092 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" event={"ID":"a69ad986-9fba-4a09-9d5f-65236575a370","Type":"ContainerStarted","Data":"b8a5a2969b48376d58611b2de5725d1bc39be0a29882a1d047e4f4a1da621656"} Oct 30 20:17:20.097291 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:20.097277 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:20.160391 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:20.160337 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" podStartSLOduration=2.160320584 podStartE2EDuration="2.160320584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:17:20.159086519 +0000 UTC m=+1756.559258526" watchObservedRunningTime="2026-10-30 20:17:20.160320584 +0000 UTC m=+1756.560492593" Oct 30 20:17:32.107797 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:17:32.107764 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-wxz46" Oct 30 20:17:35.090057 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:17:35.090019 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:18:05.091332 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:18:05.091293 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:18:11.346692 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.346659 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc"] Oct 30 20:18:11.347235 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.347147 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca522e5c-d8e7-4bba-b105-4a4794d37db6" containerName="istio-proxy" Oct 30 20:18:11.350319 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.350299 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.368929 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.368863 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-5d6bba-openshift-default-dockercfg-4fn6j\"" Oct 30 20:18:11.384084 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.384057 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc"] Oct 30 20:18:11.393618 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.393594 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/560a03d6-f472-4e54-8986-33064de8ac33-istiod-ca-cert\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.393726 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.393627 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-envoy\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.393726 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.393653 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-istio-token\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.393840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.393824 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/560a03d6-f472-4e54-8986-33064de8ac33-istio-podinfo\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.393935 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.393920 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-socket\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.393989 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.393951 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-data\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.393989 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.393979 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-credential-socket\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.394109 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.394092 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-certs\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.394153 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.394118 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn7lf\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-kube-api-access-gn7lf\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.494850 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.494811 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-istio-token\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.495035 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.495001 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/560a03d6-f472-4e54-8986-33064de8ac33-istio-podinfo\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.495094 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.495051 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-socket\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.495094 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.495087 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-data\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.495203 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.495127 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-credential-socket\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.495256 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.495211 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-certs\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.495256 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.495236 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn7lf\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-kube-api-access-gn7lf\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.495353 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.495279 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/560a03d6-f472-4e54-8986-33064de8ac33-istiod-ca-cert\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.495353 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.495310 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-envoy\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.496221 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.496192 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-certs\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.496336 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.496298 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-credential-socket\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.496600 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.496571 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/560a03d6-f472-4e54-8986-33064de8ac33-istiod-ca-cert\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.496730 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.496711 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-data\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.496953 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.496933 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-socket\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.498269 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.498238 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-envoy\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.498804 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.498774 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/560a03d6-f472-4e54-8986-33064de8ac33-istio-podinfo\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.520964 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.520937 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-istio-token\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.526297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.526263 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn7lf\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-kube-api-access-gn7lf\") pod \"e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.662977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.662949 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:11.975954 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.975926 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc"] Oct 30 20:18:11.978483 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:18:11.978456 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560a03d6_f472_4e54_8986_33064de8ac33.slice/crio-d9e90fe57557199155bf7ebe585f65ad31e1588bec3e6ba99546a615f2882110 WatchSource:0}: Error finding container d9e90fe57557199155bf7ebe585f65ad31e1588bec3e6ba99546a615f2882110: Status 404 returned error can't find the container with id d9e90fe57557199155bf7ebe585f65ad31e1588bec3e6ba99546a615f2882110 Oct 30 20:18:11.980600 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.980569 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:18:11.980705 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.980640 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:18:11.980705 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:11.980683 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:18:12.347263 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:12.347175 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" event={"ID":"560a03d6-f472-4e54-8986-33064de8ac33","Type":"ContainerStarted","Data":"2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f"} Oct 30 20:18:12.347662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:12.347277 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" event={"ID":"560a03d6-f472-4e54-8986-33064de8ac33","Type":"ContainerStarted","Data":"d9e90fe57557199155bf7ebe585f65ad31e1588bec3e6ba99546a615f2882110"} Oct 30 20:18:12.421246 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:12.421194 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" podStartSLOduration=1.421181011 podStartE2EDuration="1.421181011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:12.40732519 +0000 UTC m=+1808.807497198" watchObservedRunningTime="2026-10-30 20:18:12.421181011 +0000 UTC m=+1808.821353065" Oct 30 20:18:12.664011 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:12.663978 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:13.668406 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:13.668379 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:14.359030 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:14.358998 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:14.359825 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:14.359807 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:16.810830 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.810791 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7"] Oct 30 20:18:16.814669 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.814645 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m"] Oct 30 20:18:16.814818 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.814760 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:16.817980 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.817958 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:16.838143 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.838118 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-5d6bba\"" Oct 30 20:18:16.842666 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.842640 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7"] Oct 30 20:18:16.848468 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.848417 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-5d6bba-serving-cert\"" Oct 30 20:18:16.848781 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.848759 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m"] Oct 30 20:18:16.849421 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.849364 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-bb58bc79-dockercfg-k9b8j\"" Oct 30 20:18:16.949958 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.949886 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189db4b-0d09-4be2-b888-2986809aa8eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7\" (UID: \"3189db4b-0d09-4be2-b888-2986809aa8eb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:16.950136 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.950024 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cacf1b61-c26f-43c0-9436-81a030c232fd-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:16.950136 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.950052 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:16.950136 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.950071 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf6vq\" (UniqueName: \"kubernetes.io/projected/cacf1b61-c26f-43c0-9436-81a030c232fd-kube-api-access-zf6vq\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:16.950280 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.950182 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zds6\" (UniqueName: \"kubernetes.io/projected/3189db4b-0d09-4be2-b888-2986809aa8eb-kube-api-access-7zds6\") pod \"payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7\" (UID: \"3189db4b-0d09-4be2-b888-2986809aa8eb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:16.950280 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:16.950222 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.050756 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.050726 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189db4b-0d09-4be2-b888-2986809aa8eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7\" (UID: \"3189db4b-0d09-4be2-b888-2986809aa8eb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:17.050966 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.050812 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cacf1b61-c26f-43c0-9436-81a030c232fd-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.050966 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.050840 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.050966 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.050864 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf6vq\" (UniqueName: \"kubernetes.io/projected/cacf1b61-c26f-43c0-9436-81a030c232fd-kube-api-access-zf6vq\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.050966 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.050926 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zds6\" (UniqueName: \"kubernetes.io/projected/3189db4b-0d09-4be2-b888-2986809aa8eb-kube-api-access-7zds6\") pod \"payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7\" (UID: \"3189db4b-0d09-4be2-b888-2986809aa8eb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:17.051212 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.051100 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.051537 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.051510 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cacf1b61-c26f-43c0-9436-81a030c232fd-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.051800 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.051613 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189db4b-0d09-4be2-b888-2986809aa8eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7\" (UID: \"3189db4b-0d09-4be2-b888-2986809aa8eb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:17.053550 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.053531 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.053646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.053559 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.081426 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.081367 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zds6\" (UniqueName: \"kubernetes.io/projected/3189db4b-0d09-4be2-b888-2986809aa8eb-kube-api-access-7zds6\") pod \"payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7\" (UID: \"3189db4b-0d09-4be2-b888-2986809aa8eb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:17.081754 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.081733 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf6vq\" (UniqueName: \"kubernetes.io/projected/cacf1b61-c26f-43c0-9436-81a030c232fd-kube-api-access-zf6vq\") pod \"maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.129981 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.129953 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:17.136037 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.136017 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:17.443652 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.443624 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7"] Oct 30 20:18:17.447551 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:18:17.447528 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3189db4b_0d09_4be2_b888_2986809aa8eb.slice/crio-52ca87927c9ae5dd9951a8c58a0781f58c19b69ba3e3d7c16bf3c0739c8b1b8f WatchSource:0}: Error finding container 52ca87927c9ae5dd9951a8c58a0781f58c19b69ba3e3d7c16bf3c0739c8b1b8f: Status 404 returned error can't find the container with id 52ca87927c9ae5dd9951a8c58a0781f58c19b69ba3e3d7c16bf3c0739c8b1b8f Oct 30 20:18:17.454115 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:17.454088 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m"] Oct 30 20:18:17.457459 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:18:17.457434 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcacf1b61_c26f_43c0_9436_81a030c232fd.slice/crio-ca6f7c223b95f2278835aeb5654633cddc1e67a51dd4d0256c604c08d5ebb029 WatchSource:0}: Error finding container ca6f7c223b95f2278835aeb5654633cddc1e67a51dd4d0256c604c08d5ebb029: Status 404 returned error can't find the container with id ca6f7c223b95f2278835aeb5654633cddc1e67a51dd4d0256c604c08d5ebb029 Oct 30 20:18:18.377705 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:18.377674 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" event={"ID":"cacf1b61-c26f-43c0-9436-81a030c232fd","Type":"ContainerStarted","Data":"a49122a45202d73a7f583be05baa7208920978f6fa75734a5c3bd785efa1cb42"} Oct 30 20:18:18.378181 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:18.377725 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" event={"ID":"cacf1b61-c26f-43c0-9436-81a030c232fd","Type":"ContainerStarted","Data":"ca6f7c223b95f2278835aeb5654633cddc1e67a51dd4d0256c604c08d5ebb029"} Oct 30 20:18:18.378181 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:18.377801 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:18.379213 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:18.379190 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" event={"ID":"3189db4b-0d09-4be2-b888-2986809aa8eb","Type":"ContainerStarted","Data":"959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52"} Oct 30 20:18:18.379324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:18.379234 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" event={"ID":"3189db4b-0d09-4be2-b888-2986809aa8eb","Type":"ContainerStarted","Data":"52ca87927c9ae5dd9951a8c58a0781f58c19b69ba3e3d7c16bf3c0739c8b1b8f"} Oct 30 20:18:18.379324 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:18.379271 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:18.426138 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:18.426094 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" podStartSLOduration=2.130404623 podStartE2EDuration="2.426083085s" totalImagesPullingTime="295.678462ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:18.423511307 +0000 UTC m=+1814.823683299" watchObservedRunningTime="2026-10-30 20:18:18.426083085 +0000 UTC m=+1814.826255119" Oct 30 20:18:18.469931 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:18.469874 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" podStartSLOduration=2.469861723 podStartE2EDuration="2.469861723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:18.468702277 +0000 UTC m=+1814.868874285" watchObservedRunningTime="2026-10-30 20:18:18.469861723 +0000 UTC m=+1814.870033722" Oct 30 20:18:24.411209 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:24.411089 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:18:29.387448 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:29.387421 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:18:30.160470 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:30.160011 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": read tcp 10.133.0.2:60906->10.133.0.83:15021: read: connection reset by peer" start-of-body= Oct 30 20:18:30.160470 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:30.160093 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": read tcp 10.133.0.2:60906->10.133.0.83:15021: read: connection reset by peer" Oct 30 20:18:30.450381 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:30.450303 6411 generic.go:356] "Generic (PLEG): container finished" podID="560a03d6-f472-4e54-8986-33064de8ac33" containerID="2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f" exitCode=0 Oct 30 20:18:30.450381 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:30.450357 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" event={"ID":"560a03d6-f472-4e54-8986-33064de8ac33","Type":"ContainerDied","Data":"2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f"} Oct 30 20:18:30.450807 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:30.450720 6411 scope.go:122] "RemoveContainer" containerID="2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f" Oct 30 20:18:30.452119 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:30.452088 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:18:30.452191 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:30.452166 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:18:30.452231 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:30.452202 6411 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 30 20:18:31.457478 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:31.457442 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" event={"ID":"560a03d6-f472-4e54-8986-33064de8ac33","Type":"ContainerStarted","Data":"b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b"} Oct 30 20:18:31.663323 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:31.663288 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:31.663528 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:31.663485 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Oct 30 20:18:31.663604 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:31.663534 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Oct 30 20:18:32.664297 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:32.664266 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Oct 30 20:18:32.664693 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:32.664316 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Oct 30 20:18:33.664182 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:33.664151 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Oct 30 20:18:33.664468 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:33.664439 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Oct 30 20:18:34.663833 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:34.663806 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Oct 30 20:18:34.664073 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:34.664053 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Oct 30 20:18:35.115357 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:18:35.115273 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:18:35.663657 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:35.663624 6411 patch_prober.go:29] interesting pod/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Oct 30 20:18:35.663940 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:35.663912 6411 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Oct 30 20:18:37.026526 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:37.026484 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:37.027450 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:37.027399 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:18:37.028474 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:18:37.028449 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:19:05.093230 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:19:05.093192 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:19:13.434121 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:13.434085 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7"] Oct 30 20:19:14.530123 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:14.530088 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m"] Oct 30 20:19:14.530572 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:14.530339 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" podUID="cacf1b61-c26f-43c0-9436-81a030c232fd" containerName="maas-api" containerID="cri-o://a49122a45202d73a7f583be05baa7208920978f6fa75734a5c3bd785efa1cb42" gracePeriod=30 Oct 30 20:19:15.672644 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.672609 6411 generic.go:356] "Generic (PLEG): container finished" podID="cacf1b61-c26f-43c0-9436-81a030c232fd" containerID="a49122a45202d73a7f583be05baa7208920978f6fa75734a5c3bd785efa1cb42" exitCode=0 Oct 30 20:19:15.673021 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.672685 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" event={"ID":"cacf1b61-c26f-43c0-9436-81a030c232fd","Type":"ContainerDied","Data":"a49122a45202d73a7f583be05baa7208920978f6fa75734a5c3bd785efa1cb42"} Oct 30 20:19:15.792533 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.792504 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:19:15.868209 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.868115 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cacf1b61-c26f-43c0-9436-81a030c232fd-kube-api-access-zf6vq\" (UniqueName: \"kubernetes.io/projected/cacf1b61-c26f-43c0-9436-81a030c232fd-kube-api-access-zf6vq\") pod \"cacf1b61-c26f-43c0-9436-81a030c232fd\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " Oct 30 20:19:15.868382 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.868302 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-maas-api-tls\") pod \"cacf1b61-c26f-43c0-9436-81a030c232fd\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " Oct 30 20:19:15.868382 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.868373 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-metrics-certs\") pod \"cacf1b61-c26f-43c0-9436-81a030c232fd\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " Oct 30 20:19:15.868535 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.868431 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cacf1b61-c26f-43c0-9436-81a030c232fd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cacf1b61-c26f-43c0-9436-81a030c232fd-openshift-service-ca\") pod \"cacf1b61-c26f-43c0-9436-81a030c232fd\" (UID: \"cacf1b61-c26f-43c0-9436-81a030c232fd\") " Oct 30 20:19:15.868820 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.868793 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cacf1b61-c26f-43c0-9436-81a030c232fd-openshift-service-ca" pod "cacf1b61-c26f-43c0-9436-81a030c232fd" (UID: "cacf1b61-c26f-43c0-9436-81a030c232fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:19:15.869236 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.869205 6411 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cacf1b61-c26f-43c0-9436-81a030c232fd-openshift-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:15.870713 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.870683 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-metrics-certs" pod "cacf1b61-c26f-43c0-9436-81a030c232fd" (UID: "cacf1b61-c26f-43c0-9436-81a030c232fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:19:15.871091 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.870976 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cacf1b61-c26f-43c0-9436-81a030c232fd-kube-api-access-zf6vq" pod "cacf1b61-c26f-43c0-9436-81a030c232fd" (UID: "cacf1b61-c26f-43c0-9436-81a030c232fd"). InnerVolumeSpecName "kube-api-access-zf6vq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:19:15.871091 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.870984 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-maas-api-tls" pod "cacf1b61-c26f-43c0-9436-81a030c232fd" (UID: "cacf1b61-c26f-43c0-9436-81a030c232fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:19:15.970729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.970696 6411 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-maas-api-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:15.970729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.970728 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cacf1b61-c26f-43c0-9436-81a030c232fd-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:15.970729 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:15.970737 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf6vq\" (UniqueName: \"kubernetes.io/projected/cacf1b61-c26f-43c0-9436-81a030c232fd-kube-api-access-zf6vq\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:16.678977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:16.678945 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" event={"ID":"cacf1b61-c26f-43c0-9436-81a030c232fd","Type":"ContainerDied","Data":"ca6f7c223b95f2278835aeb5654633cddc1e67a51dd4d0256c604c08d5ebb029"} Oct 30 20:19:16.678977 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:16.678973 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m" Oct 30 20:19:16.679468 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:16.679006 6411 scope.go:122] "RemoveContainer" containerID="a49122a45202d73a7f583be05baa7208920978f6fa75734a5c3bd785efa1cb42" Oct 30 20:19:16.735650 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:16.735623 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m"] Oct 30 20:19:16.748241 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:16.748209 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5d6bba-5cb9d89865-l6n7m"] Oct 30 20:19:17.732282 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:17.732251 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cacf1b61-c26f-43c0-9436-81a030c232fd" path="/var/lib/kubelet/pods/cacf1b61-c26f-43c0-9436-81a030c232fd/volumes" Oct 30 20:19:18.434801 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:18.434752 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" podUID="3189db4b-0d09-4be2-b888-2986809aa8eb" containerName="payload-pre-processing" containerID="cri-o://959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52" gracePeriod=25 Oct 30 20:19:19.385767 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:19.385718 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" podUID="3189db4b-0d09-4be2-b888-2986809aa8eb" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 20:19:22.689148 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:22.689110 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc"] Oct 30 20:19:22.691758 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:22.691719 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" containerID="cri-o://b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b" gracePeriod=30 Oct 30 20:19:26.752962 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.752924 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9"] Oct 30 20:19:26.753666 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.753633 6411 cpu_manager.go:414] "containerMap: removing container" podUID="cacf1b61-c26f-43c0-9436-81a030c232fd" containerName="maas-api" Oct 30 20:19:26.753804 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.753670 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cacf1b61-c26f-43c0-9436-81a030c232fd" containerName="maas-api" Oct 30 20:19:26.753804 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.753790 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="cacf1b61-c26f-43c0-9436-81a030c232fd" containerName="maas-api" Oct 30 20:19:26.757404 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.757379 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.777535 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.777509 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9"] Oct 30 20:19:26.788388 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.788365 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-376d5a58-kserve-self-signed-certs\"" Oct 30 20:19:26.888136 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.888096 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-dshm\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.888318 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.888147 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-model-cache\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.888318 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.888306 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ee01a56-f522-40ea-a29d-07b61c049f22-tls-certs\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.888454 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.888354 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-tmp-dir\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.888581 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.888562 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5kpz\" (UniqueName: \"kubernetes.io/projected/7ee01a56-f522-40ea-a29d-07b61c049f22-kube-api-access-m5kpz\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.888628 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.888606 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-home\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.989911 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.989876 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5kpz\" (UniqueName: \"kubernetes.io/projected/7ee01a56-f522-40ea-a29d-07b61c049f22-kube-api-access-m5kpz\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.990116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.989947 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-home\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.990116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.989992 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-dshm\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.990116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.990013 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-model-cache\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.990116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.990082 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ee01a56-f522-40ea-a29d-07b61c049f22-tls-certs\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.990116 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.990111 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-tmp-dir\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.990463 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.990443 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-home\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.990518 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.990456 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-model-cache\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.990556 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.990516 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-tmp-dir\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.992366 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.992337 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-dshm\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:26.992841 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:26.992818 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ee01a56-f522-40ea-a29d-07b61c049f22-tls-certs\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:27.019658 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:27.019592 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5kpz\" (UniqueName: \"kubernetes.io/projected/7ee01a56-f522-40ea-a29d-07b61c049f22-kube-api-access-m5kpz\") pod \"e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:27.071485 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:27.071451 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:27.376370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:27.376263 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9"] Oct 30 20:19:27.383262 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:19:27.383233 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ee01a56_f522_40ea_a29d_07b61c049f22.slice/crio-b80307d112db49c5003f5f10a0990eb0f3c9da057eb7f783c2bcea809f73d5b5 WatchSource:0}: Error finding container b80307d112db49c5003f5f10a0990eb0f3c9da057eb7f783c2bcea809f73d5b5: Status 404 returned error can't find the container with id b80307d112db49c5003f5f10a0990eb0f3c9da057eb7f783c2bcea809f73d5b5 Oct 30 20:19:27.733296 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:27.733261 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" event={"ID":"7ee01a56-f522-40ea-a29d-07b61c049f22","Type":"ContainerStarted","Data":"533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864"} Oct 30 20:19:27.733441 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:27.733308 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" event={"ID":"7ee01a56-f522-40ea-a29d-07b61c049f22","Type":"ContainerStarted","Data":"b80307d112db49c5003f5f10a0990eb0f3c9da057eb7f783c2bcea809f73d5b5"} Oct 30 20:19:27.778271 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:27.777738 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" podStartSLOduration=1.777719445 podStartE2EDuration="1.777719445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:19:27.777190374 +0000 UTC m=+1884.177362383" watchObservedRunningTime="2026-10-30 20:19:27.777719445 +0000 UTC m=+1884.177891453" Oct 30 20:19:27.951580 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:27.951556 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:19:28.072403 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.072371 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:28.104561 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104527 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-envoy\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.104561 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104567 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/560a03d6-f472-4e54-8986-33064de8ac33-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/560a03d6-f472-4e54-8986-33064de8ac33-istiod-ca-cert\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.104791 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104609 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/560a03d6-f472-4e54-8986-33064de8ac33-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/560a03d6-f472-4e54-8986-33064de8ac33-istio-podinfo\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.104791 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104654 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-credential-socket\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.104791 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104681 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-istio-token\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-istio-token\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.104791 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104758 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-kube-api-access-gn7lf\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-kube-api-access-gn7lf\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.104791 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104782 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-certs\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.105065 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104829 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-socket\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.105065 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.104877 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-data\") pod \"560a03d6-f472-4e54-8986-33064de8ac33\" (UID: \"560a03d6-f472-4e54-8986-33064de8ac33\") " Oct 30 20:19:28.105065 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105031 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-credential-socket" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:19:28.105065 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105037 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/560a03d6-f472-4e54-8986-33064de8ac33-istiod-ca-cert" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:19:28.105255 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105208 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-certs" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:19:28.105318 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105272 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-data" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:19:28.105318 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105284 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-socket" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:19:28.105426 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105398 6411 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.105426 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105416 6411 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-data\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.105538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105433 6411 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/560a03d6-f472-4e54-8986-33064de8ac33-istiod-ca-cert\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.105538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105447 6411 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-credential-socket\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.105538 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.105461 6411 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-workload-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.107216 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.107154 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-istio-token" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:19:28.107216 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.107153 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-envoy" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:19:28.107431 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.107414 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/560a03d6-f472-4e54-8986-33064de8ac33-istio-podinfo" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 20:19:28.107530 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.107505 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-kube-api-access-gn7lf" pod "560a03d6-f472-4e54-8986-33064de8ac33" (UID: "560a03d6-f472-4e54-8986-33064de8ac33"). InnerVolumeSpecName "kube-api-access-gn7lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:19:28.206173 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.206136 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn7lf\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-kube-api-access-gn7lf\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.206173 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.206168 6411 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/560a03d6-f472-4e54-8986-33064de8ac33-istio-envoy\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.206369 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.206181 6411 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/560a03d6-f472-4e54-8986-33064de8ac33-istio-podinfo\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.206369 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.206192 6411 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/560a03d6-f472-4e54-8986-33064de8ac33-istio-token\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:28.740419 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.740381 6411 generic.go:356] "Generic (PLEG): container finished" podID="560a03d6-f472-4e54-8986-33064de8ac33" containerID="b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b" exitCode=0 Oct 30 20:19:28.740595 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.740430 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" Oct 30 20:19:28.740595 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.740463 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" event={"ID":"560a03d6-f472-4e54-8986-33064de8ac33","Type":"ContainerDied","Data":"b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b"} Oct 30 20:19:28.740595 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.740513 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc" event={"ID":"560a03d6-f472-4e54-8986-33064de8ac33","Type":"ContainerDied","Data":"d9e90fe57557199155bf7ebe585f65ad31e1588bec3e6ba99546a615f2882110"} Oct 30 20:19:28.740595 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.740535 6411 scope.go:122] "RemoveContainer" containerID="b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b" Oct 30 20:19:28.749799 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.749782 6411 scope.go:122] "RemoveContainer" containerID="2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f" Oct 30 20:19:28.757469 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.757449 6411 scope.go:122] "RemoveContainer" containerID="b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b" Oct 30 20:19:28.757730 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:19:28.757710 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b\": container with ID starting with b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b not found: ID does not exist" containerID="b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b" Oct 30 20:19:28.757802 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.757741 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b"} err="failed to get container status \"b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b\": rpc error: code = NotFound desc = could not find container \"b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b\": container with ID starting with b1071cd23d07509c0615c8e91aeb480ee2c79e4ae8a0356bf73cbb0824285b3b not found: ID does not exist" Oct 30 20:19:28.757802 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.757768 6411 scope.go:122] "RemoveContainer" containerID="2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f" Oct 30 20:19:28.758129 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:19:28.758108 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f\": container with ID starting with 2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f not found: ID does not exist" containerID="2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f" Oct 30 20:19:28.758175 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.758137 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f"} err="failed to get container status \"2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f\": rpc error: code = NotFound desc = could not find container \"2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f\": container with ID starting with 2b909d11ac8eee6ed926f46579430ccb91084b2fa22d08824962b7bdc5e5227f not found: ID does not exist" Oct 30 20:19:28.814107 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.814073 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc"] Oct 30 20:19:28.826249 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:28.826218 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5d6bba-openshift-default-584db7dfdfggsgc"] Oct 30 20:19:29.385543 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:29.385504 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" podUID="3189db4b-0d09-4be2-b888-2986809aa8eb" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 20:19:29.735467 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:29.735375 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="560a03d6-f472-4e54-8986-33064de8ac33" path="/var/lib/kubelet/pods/560a03d6-f472-4e54-8986-33064de8ac33/volumes" Oct 30 20:19:35.090789 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:19:35.090755 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:19:37.072108 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:37.072058 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:37.084653 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:37.084627 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:37.804707 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:37.804678 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:19:38.679810 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.679784 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:19:38.732062 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.731993 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3189db4b-0d09-4be2-b888-2986809aa8eb-kube-api-access-7zds6\" (UniqueName: \"kubernetes.io/projected/3189db4b-0d09-4be2-b888-2986809aa8eb-kube-api-access-7zds6\") pod \"3189db4b-0d09-4be2-b888-2986809aa8eb\" (UID: \"3189db4b-0d09-4be2-b888-2986809aa8eb\") " Oct 30 20:19:38.732208 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.732128 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3189db4b-0d09-4be2-b888-2986809aa8eb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189db4b-0d09-4be2-b888-2986809aa8eb-plugins-config-volume\") pod \"3189db4b-0d09-4be2-b888-2986809aa8eb\" (UID: \"3189db4b-0d09-4be2-b888-2986809aa8eb\") " Oct 30 20:19:38.732574 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.732548 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3189db4b-0d09-4be2-b888-2986809aa8eb-plugins-config-volume" pod "3189db4b-0d09-4be2-b888-2986809aa8eb" (UID: "3189db4b-0d09-4be2-b888-2986809aa8eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:19:38.734380 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.734357 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3189db4b-0d09-4be2-b888-2986809aa8eb-kube-api-access-7zds6" pod "3189db4b-0d09-4be2-b888-2986809aa8eb" (UID: "3189db4b-0d09-4be2-b888-2986809aa8eb"). InnerVolumeSpecName "kube-api-access-7zds6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:19:38.798630 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.798602 6411 generic.go:356] "Generic (PLEG): container finished" podID="3189db4b-0d09-4be2-b888-2986809aa8eb" containerID="959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52" exitCode=0 Oct 30 20:19:38.798745 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.798649 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" Oct 30 20:19:38.798745 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.798687 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" event={"ID":"3189db4b-0d09-4be2-b888-2986809aa8eb","Type":"ContainerDied","Data":"959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52"} Oct 30 20:19:38.798836 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.798746 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7" event={"ID":"3189db4b-0d09-4be2-b888-2986809aa8eb","Type":"ContainerDied","Data":"52ca87927c9ae5dd9951a8c58a0781f58c19b69ba3e3d7c16bf3c0739c8b1b8f"} Oct 30 20:19:38.798836 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.798774 6411 scope.go:122] "RemoveContainer" containerID="959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52" Oct 30 20:19:38.807493 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.807473 6411 scope.go:122] "RemoveContainer" containerID="959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52" Oct 30 20:19:38.807761 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:19:38.807740 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52\": container with ID starting with 959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52 not found: ID does not exist" containerID="959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52" Oct 30 20:19:38.807825 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.807768 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52"} err="failed to get container status \"959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52\": rpc error: code = NotFound desc = could not find container \"959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52\": container with ID starting with 959a98969ffcba8ee4df9691ab10b82594710a83eb0db0256a8cd219c19b3b52 not found: ID does not exist" Oct 30 20:19:38.832850 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.832824 6411 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189db4b-0d09-4be2-b888-2986809aa8eb-plugins-config-volume\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:38.832951 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.832852 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zds6\" (UniqueName: \"kubernetes.io/projected/3189db4b-0d09-4be2-b888-2986809aa8eb-kube-api-access-7zds6\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:19:38.854428 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.854402 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7"] Oct 30 20:19:38.861536 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:38.861516 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5d6bba-546d549kv7"] Oct 30 20:19:39.731998 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:39.731963 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3189db4b-0d09-4be2-b888-2986809aa8eb" path="/var/lib/kubelet/pods/3189db4b-0d09-4be2-b888-2986809aa8eb/volumes" Oct 30 20:19:43.335621 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.335586 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct"] Oct 30 20:19:43.336022 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.335973 6411 cpu_manager.go:414] "containerMap: removing container" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" Oct 30 20:19:43.336022 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.335985 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" Oct 30 20:19:43.336022 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.335999 6411 cpu_manager.go:414] "containerMap: removing container" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" Oct 30 20:19:43.336022 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.336004 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" Oct 30 20:19:43.336162 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.336038 6411 cpu_manager.go:414] "containerMap: removing container" podUID="3189db4b-0d09-4be2-b888-2986809aa8eb" containerName="payload-pre-processing" Oct 30 20:19:43.336162 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.336044 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3189db4b-0d09-4be2-b888-2986809aa8eb" containerName="payload-pre-processing" Oct 30 20:19:43.336162 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.336093 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" Oct 30 20:19:43.336162 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.336101 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="560a03d6-f472-4e54-8986-33064de8ac33" containerName="istio-proxy" Oct 30 20:19:43.336162 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.336111 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="3189db4b-0d09-4be2-b888-2986809aa8eb" containerName="payload-pre-processing" Oct 30 20:19:43.340711 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.340688 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.375045 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.375014 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-376d5a58-kserve-self-signed-certs\"" Oct 30 20:19:43.377320 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.377294 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct"] Oct 30 20:19:43.378502 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.378467 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-model-cache\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.378609 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.378535 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfdpj\" (UniqueName: \"kubernetes.io/projected/15fc3c4a-c0ad-4a77-810a-4099029792e7-kube-api-access-cfdpj\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.378679 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.378652 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-tmp-dir\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.378735 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.378698 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-dshm\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.378807 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.378732 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15fc3c4a-c0ad-4a77-810a-4099029792e7-tls-certs\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.378865 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.378841 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-home\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.479770 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.479734 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15fc3c4a-c0ad-4a77-810a-4099029792e7-tls-certs\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.480018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.479786 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-home\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.480018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.479849 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-model-cache\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.480018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.479930 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfdpj\" (UniqueName: \"kubernetes.io/projected/15fc3c4a-c0ad-4a77-810a-4099029792e7-kube-api-access-cfdpj\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.480018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.479969 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-tmp-dir\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.480018 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.479987 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-dshm\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.480285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.480256 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-home\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.480285 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.480278 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-model-cache\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.480455 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.480432 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-tmp-dir\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.482225 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.482201 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-dshm\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.482518 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.482498 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15fc3c4a-c0ad-4a77-810a-4099029792e7-tls-certs\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.504760 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.504740 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfdpj\" (UniqueName: \"kubernetes.io/projected/15fc3c4a-c0ad-4a77-810a-4099029792e7-kube-api-access-cfdpj\") pod \"e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.655464 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.655429 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:43.956314 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:43.956235 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct"] Oct 30 20:19:43.959406 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:19:43.959381 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15fc3c4a_c0ad_4a77_810a_4099029792e7.slice/crio-b4245207801a795e99d369425025991e33d00bea1866be13643c2e65547b25b9 WatchSource:0}: Error finding container b4245207801a795e99d369425025991e33d00bea1866be13643c2e65547b25b9: Status 404 returned error can't find the container with id b4245207801a795e99d369425025991e33d00bea1866be13643c2e65547b25b9 Oct 30 20:19:44.829617 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:44.829579 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" event={"ID":"15fc3c4a-c0ad-4a77-810a-4099029792e7","Type":"ContainerStarted","Data":"473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122"} Oct 30 20:19:44.830114 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:44.829627 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" event={"ID":"15fc3c4a-c0ad-4a77-810a-4099029792e7","Type":"ContainerStarted","Data":"b4245207801a795e99d369425025991e33d00bea1866be13643c2e65547b25b9"} Oct 30 20:19:44.876402 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:44.876341 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" podStartSLOduration=1.8763165480000001 podStartE2EDuration="1.876316548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:19:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:19:44.875951867 +0000 UTC m=+1901.276123877" watchObservedRunningTime="2026-10-30 20:19:44.876316548 +0000 UTC m=+1901.276488556" Oct 30 20:19:45.655627 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:45.655586 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:49.484922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:49.483999 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kplbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:19:49.484922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:49.484065 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kplbv" podUID="b0a807a8-3f3a-42b0-8fa5-48841828bebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:19:49.552053 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:49.551999 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-smpxp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:19:49.552379 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:49.552327 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-smpxp" podUID="f2139ca9-f068-4350-8d67-00ce2a968911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:19:53.656112 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:53.656076 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:53.668823 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:53.668800 6411 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:19:53.897743 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:19:53.897713 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:20:05.091234 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:20:05.091199 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:20:10.068088 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:10.068040 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:20:10.068580 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:10.068542 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:20:19.587615 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:19.587581 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9"] Oct 30 20:20:22.943934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:22.928666 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct"] Oct 30 20:20:34.607742 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.607683 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" podUID="7ee01a56-f522-40ea-a29d-07b61c049f22" containerName="main" containerID="cri-o://533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864" gracePeriod=45 Oct 30 20:20:34.855746 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.855724 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:20:34.918998 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.918967 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-home\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-home\") pod \"7ee01a56-f522-40ea-a29d-07b61c049f22\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " Oct 30 20:20:34.919142 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919048 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-dshm\") pod \"7ee01a56-f522-40ea-a29d-07b61c049f22\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " Oct 30 20:20:34.919142 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919122 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ee01a56-f522-40ea-a29d-07b61c049f22-tls-certs\" (UniqueName: \"kubernetes.io/secret/7ee01a56-f522-40ea-a29d-07b61c049f22-tls-certs\") pod \"7ee01a56-f522-40ea-a29d-07b61c049f22\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " Oct 30 20:20:34.919271 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919180 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ee01a56-f522-40ea-a29d-07b61c049f22-kube-api-access-m5kpz\" (UniqueName: \"kubernetes.io/projected/7ee01a56-f522-40ea-a29d-07b61c049f22-kube-api-access-m5kpz\") pod \"7ee01a56-f522-40ea-a29d-07b61c049f22\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " Oct 30 20:20:34.919271 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919246 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-model-cache\") pod \"7ee01a56-f522-40ea-a29d-07b61c049f22\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " Oct 30 20:20:34.919271 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919245 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-home" pod "7ee01a56-f522-40ea-a29d-07b61c049f22" (UID: "7ee01a56-f522-40ea-a29d-07b61c049f22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:20:34.919428 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919293 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-tmp-dir\") pod \"7ee01a56-f522-40ea-a29d-07b61c049f22\" (UID: \"7ee01a56-f522-40ea-a29d-07b61c049f22\") " Oct 30 20:20:34.919532 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919513 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-model-cache" pod "7ee01a56-f522-40ea-a29d-07b61c049f22" (UID: "7ee01a56-f522-40ea-a29d-07b61c049f22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:20:34.919640 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919626 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:34.919699 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919652 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:34.919699 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.919620 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-tmp-dir" pod "7ee01a56-f522-40ea-a29d-07b61c049f22" (UID: "7ee01a56-f522-40ea-a29d-07b61c049f22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:20:34.921481 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.921450 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ee01a56-f522-40ea-a29d-07b61c049f22-kube-api-access-m5kpz" pod "7ee01a56-f522-40ea-a29d-07b61c049f22" (UID: "7ee01a56-f522-40ea-a29d-07b61c049f22"). InnerVolumeSpecName "kube-api-access-m5kpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:20:34.921578 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.921545 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ee01a56-f522-40ea-a29d-07b61c049f22-tls-certs" pod "7ee01a56-f522-40ea-a29d-07b61c049f22" (UID: "7ee01a56-f522-40ea-a29d-07b61c049f22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:20:34.921809 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:34.921790 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-dshm" pod "7ee01a56-f522-40ea-a29d-07b61c049f22" (UID: "7ee01a56-f522-40ea-a29d-07b61c049f22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:20:35.020221 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.020192 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:35.020221 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.020219 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ee01a56-f522-40ea-a29d-07b61c049f22-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:35.020375 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.020231 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ee01a56-f522-40ea-a29d-07b61c049f22-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:35.020375 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.020242 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5kpz\" (UniqueName: \"kubernetes.io/projected/7ee01a56-f522-40ea-a29d-07b61c049f22-kube-api-access-m5kpz\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:35.091673 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:20:35.091636 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:20:35.137525 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.137490 6411 generic.go:356] "Generic (PLEG): container finished" podID="7ee01a56-f522-40ea-a29d-07b61c049f22" containerID="533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864" exitCode=0 Oct 30 20:20:35.137662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.137538 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" Oct 30 20:20:35.137662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.137539 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" event={"ID":"7ee01a56-f522-40ea-a29d-07b61c049f22","Type":"ContainerDied","Data":"533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864"} Oct 30 20:20:35.137662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.137581 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" event={"ID":"7ee01a56-f522-40ea-a29d-07b61c049f22","Type":"ContainerDied","Data":"b80307d112db49c5003f5f10a0990eb0f3c9da057eb7f783c2bcea809f73d5b5"} Oct 30 20:20:35.137662 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.137601 6411 scope.go:122] "RemoveContainer" containerID="533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864" Oct 30 20:20:35.146773 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.146753 6411 scope.go:122] "RemoveContainer" containerID="533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864" Oct 30 20:20:35.147080 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:20:35.147061 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864\": container with ID starting with 533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864 not found: ID does not exist" containerID="533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864" Oct 30 20:20:35.147152 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.147093 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864"} err="failed to get container status \"533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864\": rpc error: code = NotFound desc = could not find container \"533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864\": container with ID starting with 533855432b8ed875b4ef3615055be842a2167870706425186643eb63dce39864 not found: ID does not exist" Oct 30 20:20:35.194893 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.194816 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9"] Oct 30 20:20:35.196865 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.196839 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9"] Oct 30 20:20:35.731576 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.731547 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ee01a56-f522-40ea-a29d-07b61c049f22" path="/var/lib/kubelet/pods/7ee01a56-f522-40ea-a29d-07b61c049f22/volumes" Oct 30 20:20:35.794620 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:35.794586 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-376d5a58-kserve-7bd9c8b45-kbvj9" podUID="7ee01a56-f522-40ea-a29d-07b61c049f22" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 20:20:38.074709 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.074669 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" podUID="15fc3c4a-c0ad-4a77-810a-4099029792e7" containerName="main" containerID="cri-o://473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122" gracePeriod=45 Oct 30 20:20:38.335883 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.335809 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:20:38.348840 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.348812 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-model-cache\") pod \"15fc3c4a-c0ad-4a77-810a-4099029792e7\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " Oct 30 20:20:38.349044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.348871 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-home\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-home\") pod \"15fc3c4a-c0ad-4a77-810a-4099029792e7\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " Oct 30 20:20:38.349044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.348944 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15fc3c4a-c0ad-4a77-810a-4099029792e7-kube-api-access-cfdpj\" (UniqueName: \"kubernetes.io/projected/15fc3c4a-c0ad-4a77-810a-4099029792e7-kube-api-access-cfdpj\") pod \"15fc3c4a-c0ad-4a77-810a-4099029792e7\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " Oct 30 20:20:38.349044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.348981 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/15fc3c4a-c0ad-4a77-810a-4099029792e7-tls-certs\" (UniqueName: \"kubernetes.io/secret/15fc3c4a-c0ad-4a77-810a-4099029792e7-tls-certs\") pod \"15fc3c4a-c0ad-4a77-810a-4099029792e7\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " Oct 30 20:20:38.349044 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.349028 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-tmp-dir\") pod \"15fc3c4a-c0ad-4a77-810a-4099029792e7\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " Oct 30 20:20:38.349273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.349151 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-dshm\") pod \"15fc3c4a-c0ad-4a77-810a-4099029792e7\" (UID: \"15fc3c4a-c0ad-4a77-810a-4099029792e7\") " Oct 30 20:20:38.349531 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.349164 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-model-cache" pod "15fc3c4a-c0ad-4a77-810a-4099029792e7" (UID: "15fc3c4a-c0ad-4a77-810a-4099029792e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:20:38.349712 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.349187 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-home" pod "15fc3c4a-c0ad-4a77-810a-4099029792e7" (UID: "15fc3c4a-c0ad-4a77-810a-4099029792e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:20:38.349867 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.349702 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-tmp-dir" pod "15fc3c4a-c0ad-4a77-810a-4099029792e7" (UID: "15fc3c4a-c0ad-4a77-810a-4099029792e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:20:38.352000 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.351968 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15fc3c4a-c0ad-4a77-810a-4099029792e7-tls-certs" pod "15fc3c4a-c0ad-4a77-810a-4099029792e7" (UID: "15fc3c4a-c0ad-4a77-810a-4099029792e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:20:38.352117 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.352067 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15fc3c4a-c0ad-4a77-810a-4099029792e7-kube-api-access-cfdpj" pod "15fc3c4a-c0ad-4a77-810a-4099029792e7" (UID: "15fc3c4a-c0ad-4a77-810a-4099029792e7"). InnerVolumeSpecName "kube-api-access-cfdpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:20:38.352772 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.352744 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-dshm" pod "15fc3c4a-c0ad-4a77-810a-4099029792e7" (UID: "15fc3c4a-c0ad-4a77-810a-4099029792e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 20:20:38.464923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.460532 6411 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-dshm\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:38.464923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.460575 6411 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-model-cache\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:38.464923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.460590 6411 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-home\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:38.464923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.460609 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfdpj\" (UniqueName: \"kubernetes.io/projected/15fc3c4a-c0ad-4a77-810a-4099029792e7-kube-api-access-cfdpj\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:38.464923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.460626 6411 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15fc3c4a-c0ad-4a77-810a-4099029792e7-tls-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:38.464923 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:38.460639 6411 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15fc3c4a-c0ad-4a77-810a-4099029792e7-tmp-dir\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:20:39.158958 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.158922 6411 generic.go:356] "Generic (PLEG): container finished" podID="15fc3c4a-c0ad-4a77-810a-4099029792e7" containerID="473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122" exitCode=0 Oct 30 20:20:39.159459 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.158967 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" Oct 30 20:20:39.159459 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.159031 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" event={"ID":"15fc3c4a-c0ad-4a77-810a-4099029792e7","Type":"ContainerDied","Data":"473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122"} Oct 30 20:20:39.159459 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.159060 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct" event={"ID":"15fc3c4a-c0ad-4a77-810a-4099029792e7","Type":"ContainerDied","Data":"b4245207801a795e99d369425025991e33d00bea1866be13643c2e65547b25b9"} Oct 30 20:20:39.159459 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.159076 6411 scope.go:122] "RemoveContainer" containerID="473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122" Oct 30 20:20:39.168811 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.168790 6411 scope.go:122] "RemoveContainer" containerID="473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122" Oct 30 20:20:39.169169 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:20:39.169132 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122\": container with ID starting with 473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122 not found: ID does not exist" containerID="473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122" Oct 30 20:20:39.169222 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.169180 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122"} err="failed to get container status \"473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122\": rpc error: code = NotFound desc = could not find container \"473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122\": container with ID starting with 473d59d535f297717de9d69a0b2a74a1c6b51a8109f09111664cfb111966c122 not found: ID does not exist" Oct 30 20:20:39.225921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.225355 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct"] Oct 30 20:20:39.241921 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.236192 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-376d5a58-kserve-85c9b7d7df-jwnct"] Oct 30 20:20:39.743922 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:39.743785 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15fc3c4a-c0ad-4a77-810a-4099029792e7" path="/var/lib/kubelet/pods/15fc3c4a-c0ad-4a77-810a-4099029792e7/volumes" Oct 30 20:20:40.070144 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:40.070046 6411 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-dh2bx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 20:20:40.070416 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:20:40.070384 6411 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh2bx" podUID="3130e2e4-4b22-4b49-9a5c-542fc9b3ada9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 20:21:01.930110 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.930074 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-59sj6"] Oct 30 20:21:01.930721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.930420 6411 cpu_manager.go:414] "containerMap: removing container" podUID="15fc3c4a-c0ad-4a77-810a-4099029792e7" containerName="main" Oct 30 20:21:01.930721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.930431 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15fc3c4a-c0ad-4a77-810a-4099029792e7" containerName="main" Oct 30 20:21:01.930721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.930461 6411 cpu_manager.go:414] "containerMap: removing container" podUID="7ee01a56-f522-40ea-a29d-07b61c049f22" containerName="main" Oct 30 20:21:01.930721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.930466 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ee01a56-f522-40ea-a29d-07b61c049f22" containerName="main" Oct 30 20:21:01.930721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.930521 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ee01a56-f522-40ea-a29d-07b61c049f22" containerName="main" Oct 30 20:21:01.930721 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.930531 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="15fc3c4a-c0ad-4a77-810a-4099029792e7" containerName="main" Oct 30 20:21:01.933572 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.933551 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:01.968295 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:01.968270 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-59sj6"] Oct 30 20:21:02.016007 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.015978 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36ac6165-dfd5-4404-90cc-7c13a4074639-openshift-service-ca\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.016007 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.016016 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ac6165-dfd5-4404-90cc-7c13a4074639-maas-api-tls\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.016210 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.016036 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khzc2\" (UniqueName: \"kubernetes.io/projected/36ac6165-dfd5-4404-90cc-7c13a4074639-kube-api-access-khzc2\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.016210 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.016181 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ac6165-dfd5-4404-90cc-7c13a4074639-metrics-certs\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.117002 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.116975 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ac6165-dfd5-4404-90cc-7c13a4074639-metrics-certs\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.117192 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.117050 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36ac6165-dfd5-4404-90cc-7c13a4074639-openshift-service-ca\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.117192 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.117066 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ac6165-dfd5-4404-90cc-7c13a4074639-maas-api-tls\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.117192 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.117085 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khzc2\" (UniqueName: \"kubernetes.io/projected/36ac6165-dfd5-4404-90cc-7c13a4074639-kube-api-access-khzc2\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.117915 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.117873 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36ac6165-dfd5-4404-90cc-7c13a4074639-openshift-service-ca\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.119790 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.119767 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ac6165-dfd5-4404-90cc-7c13a4074639-metrics-certs\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.119931 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.119771 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ac6165-dfd5-4404-90cc-7c13a4074639-maas-api-tls\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.143347 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.143321 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khzc2\" (UniqueName: \"kubernetes.io/projected/36ac6165-dfd5-4404-90cc-7c13a4074639-kube-api-access-khzc2\") pod \"maas-api-595cc648c-59sj6\" (UID: \"36ac6165-dfd5-4404-90cc-7c13a4074639\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.247040 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.246973 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:02.557397 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.557349 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-59sj6"] Oct 30 20:21:02.560171 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:21:02.560140 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ac6165_dfd5_4404_90cc_7c13a4074639.slice/crio-a9731939a5855afde0bb0e9d977e059a48f5562c0f13f53aec166f1ae9dcfccc WatchSource:0}: Error finding container a9731939a5855afde0bb0e9d977e059a48f5562c0f13f53aec166f1ae9dcfccc: Status 404 returned error can't find the container with id a9731939a5855afde0bb0e9d977e059a48f5562c0f13f53aec166f1ae9dcfccc Oct 30 20:21:02.561751 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:02.561728 6411 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 20:21:03.300925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:03.300829 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" event={"ID":"36ac6165-dfd5-4404-90cc-7c13a4074639","Type":"ContainerStarted","Data":"80efc10ff802c0c23cac8d91163b191d38378069fcb2c94d121816207782459c"} Oct 30 20:21:03.300925 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:03.300884 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" event={"ID":"36ac6165-dfd5-4404-90cc-7c13a4074639","Type":"ContainerStarted","Data":"a9731939a5855afde0bb0e9d977e059a48f5562c0f13f53aec166f1ae9dcfccc"} Oct 30 20:21:03.301364 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:03.300944 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:03.349587 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:03.349533 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" podStartSLOduration=2.030496699 podStartE2EDuration="2.349517379s" totalImagesPullingTime="319.02068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:21:03.348152896 +0000 UTC m=+1979.748324912" watchObservedRunningTime="2026-10-30 20:21:03.349517379 +0000 UTC m=+1979.749689386" Oct 30 20:21:05.091051 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:21:05.091013 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:21:09.318590 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:09.318561 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-59sj6" Oct 30 20:21:09.411430 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:09.411393 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-mcct8"] Oct 30 20:21:09.411699 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:09.411669 6411 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" podUID="68d76a19-a383-41d9-8321-eb7f74680af0" containerName="maas-api" containerID="cri-o://e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419" gracePeriod=30 Oct 30 20:21:10.663888 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.663852 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 20:21:10.803674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.803594 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-maas-api-tls\") pod \"68d76a19-a383-41d9-8321-eb7f74680af0\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " Oct 30 20:21:10.803674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.803631 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68d76a19-a383-41d9-8321-eb7f74680af0-kube-api-access-pwhdr\" (UniqueName: \"kubernetes.io/projected/68d76a19-a383-41d9-8321-eb7f74680af0-kube-api-access-pwhdr\") pod \"68d76a19-a383-41d9-8321-eb7f74680af0\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " Oct 30 20:21:10.803949 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.803776 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68d76a19-a383-41d9-8321-eb7f74680af0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d76a19-a383-41d9-8321-eb7f74680af0-openshift-service-ca\") pod \"68d76a19-a383-41d9-8321-eb7f74680af0\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " Oct 30 20:21:10.803949 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.803802 6411 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-metrics-certs\") pod \"68d76a19-a383-41d9-8321-eb7f74680af0\" (UID: \"68d76a19-a383-41d9-8321-eb7f74680af0\") " Oct 30 20:21:10.804203 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.804176 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d76a19-a383-41d9-8321-eb7f74680af0-openshift-service-ca" pod "68d76a19-a383-41d9-8321-eb7f74680af0" (UID: "68d76a19-a383-41d9-8321-eb7f74680af0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 20:21:10.805964 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.805937 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-maas-api-tls" pod "68d76a19-a383-41d9-8321-eb7f74680af0" (UID: "68d76a19-a383-41d9-8321-eb7f74680af0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:21:10.806089 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.806048 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-metrics-certs" pod "68d76a19-a383-41d9-8321-eb7f74680af0" (UID: "68d76a19-a383-41d9-8321-eb7f74680af0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 20:21:10.806089 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.806068 6411 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d76a19-a383-41d9-8321-eb7f74680af0-kube-api-access-pwhdr" pod "68d76a19-a383-41d9-8321-eb7f74680af0" (UID: "68d76a19-a383-41d9-8321-eb7f74680af0"). InnerVolumeSpecName "kube-api-access-pwhdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 20:21:10.905237 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.905190 6411 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d76a19-a383-41d9-8321-eb7f74680af0-openshift-service-ca\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:21:10.905237 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.905231 6411 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-metrics-certs\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:21:10.905237 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.905245 6411 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d76a19-a383-41d9-8321-eb7f74680af0-maas-api-tls\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:21:10.905472 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:10.905258 6411 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwhdr\" (UniqueName: \"kubernetes.io/projected/68d76a19-a383-41d9-8321-eb7f74680af0-kube-api-access-pwhdr\") on node \"ip-10-0-142-168.ec2.internal\" DevicePath \"\"" Oct 30 20:21:11.345679 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.345640 6411 generic.go:356] "Generic (PLEG): container finished" podID="68d76a19-a383-41d9-8321-eb7f74680af0" containerID="e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419" exitCode=0 Oct 30 20:21:11.345890 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.345712 6411 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" Oct 30 20:21:11.345890 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.345734 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" event={"ID":"68d76a19-a383-41d9-8321-eb7f74680af0","Type":"ContainerDied","Data":"e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419"} Oct 30 20:21:11.345890 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.345812 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-mcct8" event={"ID":"68d76a19-a383-41d9-8321-eb7f74680af0","Type":"ContainerDied","Data":"8fce221f6907ee4bbfb3526dbda2071b5da8a1b583ce334fbc6741412ba2a000"} Oct 30 20:21:11.345890 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.345845 6411 scope.go:122] "RemoveContainer" containerID="e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419" Oct 30 20:21:11.356888 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.356864 6411 scope.go:122] "RemoveContainer" containerID="e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419" Oct 30 20:21:11.357689 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:21:11.357666 6411 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419\": container with ID starting with e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419 not found: ID does not exist" containerID="e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419" Oct 30 20:21:11.357801 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.357702 6411 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419"} err="failed to get container status \"e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419\": rpc error: code = NotFound desc = could not find container \"e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419\": container with ID starting with e7e927ef7e9e5955d0514944e30d2b9c4076772655c59c46ed4eccd4e1ec8419 not found: ID does not exist" Oct 30 20:21:11.404815 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.404721 6411 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-mcct8"] Oct 30 20:21:11.417365 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.417330 6411 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-mcct8"] Oct 30 20:21:11.736390 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:21:11.735997 6411 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68d76a19-a383-41d9-8321-eb7f74680af0" path="/var/lib/kubelet/pods/68d76a19-a383-41d9-8321-eb7f74680af0/volumes" Oct 30 20:21:35.093648 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:21:35.093612 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:22:01.013605 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:01.013566 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68db7997f9-9r6gr_92343fb7-b5f2-42c6-a516-aaa881456c5f/authorino/0.log" Oct 30 20:22:05.091134 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:22:05.091098 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:22:12.139198 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:12.139167 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6dd98c85c8-5rphq_e2c2db81-c0d6-4d72-9fae-24e76a66409c/manager/0.log" Oct 30 20:22:13.687747 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:13.687719 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-h8mrc_89171348-f79f-4eb0-b8be-b73a4130dc09/manager/0.log" Oct 30 20:22:14.200800 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:14.200766 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f6c7b487f-7jznk_9fc06a89-0b46-4c86-9271-4f2552ca4625/manager/0.log" Oct 30 20:22:14.261274 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:14.261252 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f6c7b487f-7jznk_9fc06a89-0b46-4c86-9271-4f2552ca4625/copy-manifests/0.log" Oct 30 20:22:16.004348 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:16.004322 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-59sj6_36ac6165-dfd5-4404-90cc-7c13a4074639/maas-api/0.log" Oct 30 20:22:19.784466 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:19.784439 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68db7997f9-9r6gr_92343fb7-b5f2-42c6-a516-aaa881456c5f/authorino/0.log" Oct 30 20:22:20.671708 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:20.671673 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-wxz46_a69ad986-9fba-4a09-9d5f-65236575a370/manager/0.log" Oct 30 20:22:22.364656 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:22.364623 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f5c96996f-sjwt9_fa5b7421-a7a7-4ea6-8d97-884649e01eac/kube-auth-proxy/0.log" Oct 30 20:22:23.178699 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:23.178667 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-dh2bx_3130e2e4-4b22-4b49-9a5c-542fc9b3ada9/istio-proxy/0.log" Oct 30 20:22:23.605934 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:23.605837 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-kplbv_b0a807a8-3f3a-42b0-8fa5-48841828bebc/istio-proxy/0.log" Oct 30 20:22:24.444638 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:24.444266 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-smpxp_f2139ca9-f068-4350-8d67-00ce2a968911/istio-proxy/0.log" Oct 30 20:22:24.865537 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:24.865464 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-xhkln_5ccf9e4e-fb38-41df-8b3d-c7210abcfb74/payload-pre-processing/0.log" Oct 30 20:22:25.067269 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:25.067239 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-6bfdff5577-vmghk_1f667352-a367-45dd-9b7b-a1b03d2d24a3/payload-processing/0.log" Oct 30 20:22:26.518273 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:26.518244 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-j9kd2_2182470d-4038-4ea2-9aaf-930c7653e2dd/main/0.log" Oct 30 20:22:27.127646 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:27.127614 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6_db9915d4-4851-42e5-bd0f-867a621026ee/main/0.log" Oct 30 20:22:27.331493 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:27.331459 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz_1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb/main/0.log" Oct 30 20:22:28.476603 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:28.476578 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68db7997f9-9r6gr_92343fb7-b5f2-42c6-a516-aaa881456c5f/authorino/0.log" Oct 30 20:22:35.090300 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:22:35.090266 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:22:38.455815 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:38.455784 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6dd98c85c8-5rphq_e2c2db81-c0d6-4d72-9fae-24e76a66409c/manager/0.log" Oct 30 20:22:39.785794 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:39.785761 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-h8mrc_89171348-f79f-4eb0-b8be-b73a4130dc09/manager/0.log" Oct 30 20:22:40.285628 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:40.285601 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f6c7b487f-7jznk_9fc06a89-0b46-4c86-9271-4f2552ca4625/manager/0.log" Oct 30 20:22:40.337992 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:40.337968 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f6c7b487f-7jznk_9fc06a89-0b46-4c86-9271-4f2552ca4625/copy-manifests/0.log" Oct 30 20:22:41.596539 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:41.596467 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-59sj6_36ac6165-dfd5-4404-90cc-7c13a4074639/maas-api/0.log" Oct 30 20:22:44.858370 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:44.858341 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68db7997f9-9r6gr_92343fb7-b5f2-42c6-a516-aaa881456c5f/authorino/0.log" Oct 30 20:22:45.717813 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:45.717780 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-wxz46_a69ad986-9fba-4a09-9d5f-65236575a370/manager/0.log" Oct 30 20:22:47.137125 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:47.137098 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f5c96996f-sjwt9_fa5b7421-a7a7-4ea6-8d97-884649e01eac/kube-auth-proxy/0.log" Oct 30 20:22:47.941334 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:47.941302 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-dh2bx_3130e2e4-4b22-4b49-9a5c-542fc9b3ada9/istio-proxy/0.log" Oct 30 20:22:48.367436 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:48.367359 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-kplbv_b0a807a8-3f3a-42b0-8fa5-48841828bebc/istio-proxy/0.log" Oct 30 20:22:49.195402 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:49.195371 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-smpxp_f2139ca9-f068-4350-8d67-00ce2a968911/istio-proxy/0.log" Oct 30 20:22:49.612556 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:49.612485 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-xhkln_5ccf9e4e-fb38-41df-8b3d-c7210abcfb74/payload-pre-processing/0.log" Oct 30 20:22:49.809773 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:49.809739 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-6bfdff5577-vmghk_1f667352-a367-45dd-9b7b-a1b03d2d24a3/payload-processing/0.log" Oct 30 20:22:50.772762 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:50.772727 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-j9kd2_2182470d-4038-4ea2-9aaf-930c7653e2dd/main/0.log" Oct 30 20:22:51.394200 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:51.394169 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8sc2p6_db9915d4-4851-42e5-bd0f-867a621026ee/main/0.log" Oct 30 20:22:51.593698 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:22:51.593673 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-vwndz_1b5f6daf-7e26-4776-ae87-aadb1a4ffcbb/main/0.log" Oct 30 20:23:05.091213 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:23:05.091172 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:23:28.688674 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:23:28.688645 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-6bfdff5577-vmghk_1f667352-a367-45dd-9b7b-a1b03d2d24a3/payload-processing/0.log" Oct 30 20:23:28.889955 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:23:28.889925 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-6bfdff5577-vmghk_1f667352-a367-45dd-9b7b-a1b03d2d24a3/payload-processing/0.log" Oct 30 20:23:35.091281 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:23:35.091244 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:24:05.091236 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:24:05.091202 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:24:13.945615 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:13.945540 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-6bfdff5577-vmghk_1f667352-a367-45dd-9b7b-a1b03d2d24a3/payload-processing/0.log" Oct 30 20:24:14.150149 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.150101 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-6bfdff5577-vmghk_1f667352-a367-45dd-9b7b-a1b03d2d24a3/payload-processing/0.log" Oct 30 20:24:14.489673 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.489643 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kc489/must-gather-l2nz9"] Oct 30 20:24:14.490082 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.490069 6411 cpu_manager.go:414] "containerMap: removing container" podUID="68d76a19-a383-41d9-8321-eb7f74680af0" containerName="maas-api" Oct 30 20:24:14.490130 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.490084 6411 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68d76a19-a383-41d9-8321-eb7f74680af0" containerName="maas-api" Oct 30 20:24:14.490166 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.490148 6411 memory_manager.go:365] "RemoveStaleState removing state" podUID="68d76a19-a383-41d9-8321-eb7f74680af0" containerName="maas-api" Oct 30 20:24:14.493270 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.493255 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kc489/must-gather-l2nz9" Oct 30 20:24:14.510545 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.510516 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kc489\"/\"openshift-service-ca.crt\"" Oct 30 20:24:14.510666 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.510622 6411 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kc489\"/\"kube-root-ca.crt\"" Oct 30 20:24:14.523139 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.523120 6411 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kc489\"/\"default-dockercfg-zvbwx\"" Oct 30 20:24:14.536152 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.536131 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kc489/must-gather-l2nz9"] Oct 30 20:24:14.612941 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.612885 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae1fee19-f81c-4682-85d2-2307554cfcb1-must-gather-output\") pod \"must-gather-l2nz9\" (UID: \"ae1fee19-f81c-4682-85d2-2307554cfcb1\") " pod="openshift-must-gather-kc489/must-gather-l2nz9" Oct 30 20:24:14.613096 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.613055 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5dp5\" (UniqueName: \"kubernetes.io/projected/ae1fee19-f81c-4682-85d2-2307554cfcb1-kube-api-access-t5dp5\") pod \"must-gather-l2nz9\" (UID: \"ae1fee19-f81c-4682-85d2-2307554cfcb1\") " pod="openshift-must-gather-kc489/must-gather-l2nz9" Oct 30 20:24:14.713892 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.713854 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5dp5\" (UniqueName: \"kubernetes.io/projected/ae1fee19-f81c-4682-85d2-2307554cfcb1-kube-api-access-t5dp5\") pod \"must-gather-l2nz9\" (UID: \"ae1fee19-f81c-4682-85d2-2307554cfcb1\") " pod="openshift-must-gather-kc489/must-gather-l2nz9" Oct 30 20:24:14.714067 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.713948 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae1fee19-f81c-4682-85d2-2307554cfcb1-must-gather-output\") pod \"must-gather-l2nz9\" (UID: \"ae1fee19-f81c-4682-85d2-2307554cfcb1\") " pod="openshift-must-gather-kc489/must-gather-l2nz9" Oct 30 20:24:14.714281 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.714260 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae1fee19-f81c-4682-85d2-2307554cfcb1-must-gather-output\") pod \"must-gather-l2nz9\" (UID: \"ae1fee19-f81c-4682-85d2-2307554cfcb1\") " pod="openshift-must-gather-kc489/must-gather-l2nz9" Oct 30 20:24:14.740477 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.740404 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5dp5\" (UniqueName: \"kubernetes.io/projected/ae1fee19-f81c-4682-85d2-2307554cfcb1-kube-api-access-t5dp5\") pod \"must-gather-l2nz9\" (UID: \"ae1fee19-f81c-4682-85d2-2307554cfcb1\") " pod="openshift-must-gather-kc489/must-gather-l2nz9" Oct 30 20:24:14.806545 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:14.806519 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kc489/must-gather-l2nz9" Oct 30 20:24:15.101606 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:15.101424 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kc489/must-gather-l2nz9"] Oct 30 20:24:15.104273 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:24:15.104242 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae1fee19_f81c_4682_85d2_2307554cfcb1.slice/crio-e27e477142e80df63527fdce1e1b5f9a1e3f5989e675f1faf559b0451c7abbf3 WatchSource:0}: Error finding container e27e477142e80df63527fdce1e1b5f9a1e3f5989e675f1faf559b0451c7abbf3: Status 404 returned error can't find the container with id e27e477142e80df63527fdce1e1b5f9a1e3f5989e675f1faf559b0451c7abbf3 Oct 30 20:24:15.226924 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:15.226878 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kc489/must-gather-l2nz9" event={"ID":"ae1fee19-f81c-4682-85d2-2307554cfcb1","Type":"ContainerStarted","Data":"e27e477142e80df63527fdce1e1b5f9a1e3f5989e675f1faf559b0451c7abbf3"} Oct 30 20:24:16.232838 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:16.232805 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kc489/must-gather-l2nz9" event={"ID":"ae1fee19-f81c-4682-85d2-2307554cfcb1","Type":"ContainerStarted","Data":"b14ae638aadd044d29a816a310ab55a0c16a8fea71c9e6de0e33ea9135b6e7e5"} Oct 30 20:24:16.233201 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:16.232845 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kc489/must-gather-l2nz9" event={"ID":"ae1fee19-f81c-4682-85d2-2307554cfcb1","Type":"ContainerStarted","Data":"c47348e549c54f6f436724f4358b4377ab90d0812eb047a039f5c926c5d9a32c"} Oct 30 20:24:20.905501 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:20.905466 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jvsfx_a3757b61-1b91-4739-a7fc-ecbfcb4feea6/global-pull-secret-syncer/0.log" Oct 30 20:24:21.569620 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:21.569588 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-44glf_3009ff95-7457-4856-be99-e3891d923002/konnectivity-agent/0.log" Oct 30 20:24:22.288694 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:22.288665 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-168.ec2.internal_41493c3824738811979a28105df74e81/haproxy/0.log" Oct 30 20:24:27.517606 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:27.517563 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68db7997f9-9r6gr_92343fb7-b5f2-42c6-a516-aaa881456c5f/authorino/0.log" Oct 30 20:24:28.027046 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:28.026992 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-wxz46_a69ad986-9fba-4a09-9d5f-65236575a370/manager/0.log" Oct 30 20:24:30.489962 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.489875 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-kc489/must-gather-l2nz9" podStartSLOduration=15.741885484 podStartE2EDuration="16.489855045s" totalImagesPullingTime="747.969561ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:24:17.295791361 +0000 UTC m=+2173.695963383" watchObservedRunningTime="2026-10-30 20:24:30.489855045 +0000 UTC m=+2186.890027050" Oct 30 20:24:30.490473 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.490299 6411 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l"] Oct 30 20:24:30.499056 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.499031 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.513723 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.513695 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l"] Oct 30 20:24:30.530796 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.530765 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7glpv\" (UniqueName: \"kubernetes.io/projected/aaf4d0a7-516b-49ad-a252-952b9170c987-kube-api-access-7glpv\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.530994 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.530819 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-proc\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.530994 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.530923 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-lib-modules\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.530994 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.530983 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-podres\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.531141 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.531057 6411 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-sys\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.632642 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632609 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7glpv\" (UniqueName: \"kubernetes.io/projected/aaf4d0a7-516b-49ad-a252-952b9170c987-kube-api-access-7glpv\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.632817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632659 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-proc\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.632817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632705 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-lib-modules\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.632817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632736 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-podres\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.632817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632761 6411 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-sys\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.632817 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632805 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-proc\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.633056 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632823 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-lib-modules\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.633056 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632841 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-podres\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.633056 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.632887 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aaf4d0a7-516b-49ad-a252-952b9170c987-sys\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.660376 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.660334 6411 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7glpv\" (UniqueName: \"kubernetes.io/projected/aaf4d0a7-516b-49ad-a252-952b9170c987-kube-api-access-7glpv\") pod \"perf-node-gather-daemonset-nnm7l\" (UID: \"aaf4d0a7-516b-49ad-a252-952b9170c987\") " pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:30.813386 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:30.813297 6411 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:31.112755 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.112717 6411 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l"] Oct 30 20:24:31.115416 ip-10-0-142-168 kubenswrapper[6411]: W1030 20:24:31.115379 6411 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaaf4d0a7_516b_49ad_a252_952b9170c987.slice/crio-02d938b901a36802e5e308bfa50064a100bcb0854ed80139f9648a7e4f80735e WatchSource:0}: Error finding container 02d938b901a36802e5e308bfa50064a100bcb0854ed80139f9648a7e4f80735e: Status 404 returned error can't find the container with id 02d938b901a36802e5e308bfa50064a100bcb0854ed80139f9648a7e4f80735e Oct 30 20:24:31.208775 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.208752 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5f9d8fd57c-dkv9w_a6f79c4a-177a-4512-b817-92404fd99b41/metrics-server/0.log" Oct 30 20:24:31.356272 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.356235 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" event={"ID":"aaf4d0a7-516b-49ad-a252-952b9170c987","Type":"ContainerStarted","Data":"ad4a12e3b724095425a4807bd544bc36b0654b8bf60c473e4979a5db3bd3d8bc"} Oct 30 20:24:31.356470 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.356297 6411 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" event={"ID":"aaf4d0a7-516b-49ad-a252-952b9170c987","Type":"ContainerStarted","Data":"02d938b901a36802e5e308bfa50064a100bcb0854ed80139f9648a7e4f80735e"} Oct 30 20:24:31.356470 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.356328 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:31.400432 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.400377 6411 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" podStartSLOduration=1.400353183 podStartE2EDuration="1.400353183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:24:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:24:31.399303548 +0000 UTC m=+2187.799475555" watchObservedRunningTime="2026-10-30 20:24:31.400353183 +0000 UTC m=+2187.800525190" Oct 30 20:24:31.456503 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.456470 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dcpkx_0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/node-exporter/0.log" Oct 30 20:24:31.579135 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.579094 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dcpkx_0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/kube-rbac-proxy/0.log" Oct 30 20:24:31.703608 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:31.703536 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dcpkx_0ba49ca5-7b3e-42bc-aa61-59004aeb73dc/init-textfile/0.log" Oct 30 20:24:32.920064 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:32.920038 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a3bbea30-36c9-4cf5-b9c6-782a0f9c6662/prometheus/0.log" Oct 30 20:24:33.042168 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:33.042123 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a3bbea30-36c9-4cf5-b9c6-782a0f9c6662/config-reloader/0.log" Oct 30 20:24:33.163404 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:33.163370 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a3bbea30-36c9-4cf5-b9c6-782a0f9c6662/thanos-sidecar/0.log" Oct 30 20:24:33.292381 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:33.292300 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a3bbea30-36c9-4cf5-b9c6-782a0f9c6662/kube-rbac-proxy-web/0.log" Oct 30 20:24:33.414551 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:33.414524 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a3bbea30-36c9-4cf5-b9c6-782a0f9c6662/kube-rbac-proxy/0.log" Oct 30 20:24:33.537645 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:33.537618 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a3bbea30-36c9-4cf5-b9c6-782a0f9c6662/kube-rbac-proxy-thanos/0.log" Oct 30 20:24:33.662261 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:33.662228 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a3bbea30-36c9-4cf5-b9c6-782a0f9c6662/init-config-reloader/0.log" Oct 30 20:24:33.795010 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:33.794982 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-xbkdh_c1873229-3e23-4ff6-9f57-f8520e9d9852/prometheus-operator/0.log" Oct 30 20:24:33.915160 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:33.915082 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-xbkdh_c1873229-3e23-4ff6-9f57-f8520e9d9852/kube-rbac-proxy/0.log" Oct 30 20:24:35.097459 ip-10-0-142-168 kubenswrapper[6411]: E1030 20:24:35.097425 6411 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-887vm" containerName="extractor" Oct 30 20:24:37.376221 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:37.376173 6411 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kc489/perf-node-gather-daemonset-nnm7l" Oct 30 20:24:38.651612 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:38.651579 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7749b9cc5c-vbb2b_e68ea6be-6fec-4347-a3f9-720743a2eaf2/console/0.log" Oct 30 20:24:42.465615 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:42.465584 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwb2s_46a18c80-179d-46ba-85aa-1489814be80e/dns/0.log" Oct 30 20:24:42.584664 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:42.584633 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwb2s_46a18c80-179d-46ba-85aa-1489814be80e/kube-rbac-proxy/0.log" Oct 30 20:24:43.428712 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:43.428682 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p6pc9_bb90a295-9822-4874-9014-0a91ac7a73c5/dns-node-resolver/0.log" Oct 30 20:24:44.794973 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:44.794920 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ghl6s_bd8ee0f0-ae69-46d4-985f-b8c9e5f5ac57/node-ca/0.log" Oct 30 20:24:45.808372 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:45.808340 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f5c96996f-sjwt9_fa5b7421-a7a7-4ea6-8d97-884649e01eac/kube-auth-proxy/0.log" Oct 30 20:24:46.063098 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:46.062986 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-dh2bx_3130e2e4-4b22-4b49-9a5c-542fc9b3ada9/istio-proxy/0.log" Oct 30 20:24:46.201316 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:46.201281 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-kplbv_b0a807a8-3f3a-42b0-8fa5-48841828bebc/istio-proxy/0.log" Oct 30 20:24:46.475984 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:46.475955 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-smpxp_f2139ca9-f068-4350-8d67-00ce2a968911/istio-proxy/0.log" Oct 30 20:24:46.614005 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:46.613975 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-xhkln_5ccf9e4e-fb38-41df-8b3d-c7210abcfb74/payload-pre-processing/0.log" Oct 30 20:24:46.683293 ip-10-0-142-168 kubenswrapper[6411]: I1030 20:24:46.683264 6411 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-6bfdff5577-vmghk_1f667352-a367-45dd-9b7b-a1b03d2d24a3/payload-processing/0.log"