Oct 30 19:18:28.292979 ip-10-0-139-211 systemd[1]: Starting Kubernetes Kubelet... Oct 30 19:18:28.727424 ip-10-0-139-211 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:18:28.727424 ip-10-0-139-211 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 19:18:28.727424 ip-10-0-139-211 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:18:28.727424 ip-10-0-139-211 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:18:28.731646 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731553 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:18:28.731646 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731642 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:18:28.731646 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731646 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:18:28.731646 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731650 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:18:28.731646 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731653 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:18:28.731646 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731656 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731658 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731662 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731666 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731671 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731674 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731677 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731680 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731682 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731686 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731690 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731693 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731697 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731716 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731719 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731722 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731725 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731728 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731730 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:18:28.731884 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731733 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731735 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731738 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731741 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731743 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731746 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731748 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731757 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731764 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731767 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731770 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731773 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731776 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731778 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731781 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731784 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731787 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731789 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731792 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731794 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:18:28.732327 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731797 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731799 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731802 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731805 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731808 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731810 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731813 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731816 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731818 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731821 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731823 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731826 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731828 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731830 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731833 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731835 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731838 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731841 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731843 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731846 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:18:28.732819 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731849 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731852 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731854 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731857 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731860 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731862 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731865 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731867 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731870 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731872 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731875 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731877 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731880 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731883 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731886 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731889 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731891 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731894 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731897 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:18:28.733276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731900 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731903 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731906 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731908 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731911 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731913 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731916 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731919 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731921 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731924 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731927 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731929 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731932 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731935 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731939 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731941 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731944 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731947 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731949 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731952 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:18:28.733728 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731955 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731958 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.731961 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732765 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732772 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732775 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732778 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732780 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732783 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732785 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732788 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732790 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732793 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732796 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732799 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732802 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732805 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732807 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732810 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:18:28.734187 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732812 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732815 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732817 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732822 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732825 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732828 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732830 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732833 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732836 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732839 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732842 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732844 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732847 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732849 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732852 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732855 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732857 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732860 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732862 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732865 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:18:28.734639 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732867 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732869 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732872 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732874 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732877 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732879 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732882 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732885 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732888 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732891 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732893 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732895 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732898 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732900 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732903 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732905 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732908 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732912 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732915 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732919 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:18:28.735132 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732921 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732924 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732926 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732930 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732935 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732938 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732941 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732944 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732946 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732949 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732951 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732954 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732956 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732959 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732961 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732963 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732966 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732968 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732971 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:18:28.735605 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732973 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732976 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732979 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732981 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732984 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732986 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732989 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732991 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732994 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.732997 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733000 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733002 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733009 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733011 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733014 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733016 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733019 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733021 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733024 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:18:28.736090 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733026 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733029 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733031 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733034 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733036 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733039 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733041 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733044 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733046 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733048 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733051 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733053 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733550 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733555 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733559 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733562 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733565 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733567 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733569 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733572 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:18:28.736533 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733575 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733578 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733580 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733583 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733585 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733590 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733592 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733595 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733597 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733600 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733602 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733605 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733607 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733610 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733612 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733615 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733617 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733619 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733622 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733625 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:18:28.737013 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733627 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733629 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733632 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733634 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733637 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733639 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733642 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733645 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733647 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733650 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733652 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733655 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733657 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733660 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733663 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733665 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733668 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733672 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733674 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733677 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:18:28.737508 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733679 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733682 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733685 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733687 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733690 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733693 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733695 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733698 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733715 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733718 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733720 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733722 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733725 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733728 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733730 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733733 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733735 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733738 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733740 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733743 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:18:28.737996 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733746 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733749 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733751 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733754 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733757 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733759 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733762 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733764 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733767 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733771 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733773 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733776 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733778 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733781 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733783 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733786 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733789 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733791 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733794 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:18:28.738460 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733796 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733799 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733801 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733804 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733806 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733808 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733811 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733814 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733816 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733818 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733821 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733825 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733829 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733832 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733836 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733841 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733844 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733848 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.733850 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:18:28.738954 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734823 2579 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734834 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734840 2579 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734846 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734850 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734854 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734861 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734865 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734869 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734871 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734875 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734878 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734881 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734884 2579 flags.go:64] FLAG: --cgroup-root="" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734887 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734889 2579 flags.go:64] FLAG: --client-ca-file="" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734892 2579 flags.go:64] FLAG: --cloud-provider="external" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734896 2579 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734901 2579 flags.go:64] FLAG: --cluster-domain="" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734904 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734907 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734910 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734913 2579 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734921 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 19:18:28.739403 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734924 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734927 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734930 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734933 2579 flags.go:64] FLAG: --contention-profiling="false" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734938 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734941 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734944 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734947 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734951 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734954 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734957 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734960 2579 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734963 2579 flags.go:64] FLAG: --enable-server="true" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734966 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734970 2579 flags.go:64] FLAG: --event-burst="100" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734973 2579 flags.go:64] FLAG: --event-qps="50" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734976 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734979 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734982 2579 flags.go:64] FLAG: --eviction-hard="" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734986 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734989 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734992 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734995 2579 flags.go:64] FLAG: --eviction-soft="" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.734998 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735001 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 19:18:28.739971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735004 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735006 2579 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735009 2579 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735012 2579 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735017 2579 flags.go:64] FLAG: --feature-gates="" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735021 2579 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735024 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735027 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735030 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735033 2579 flags.go:64] FLAG: --healthz-port="10248" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735036 2579 flags.go:64] FLAG: --help="false" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735039 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735042 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735045 2579 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735048 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735051 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735055 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735058 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735061 2579 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735063 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735066 2579 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735069 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735072 2579 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735083 2579 flags.go:64] FLAG: --kube-reserved="" Oct 30 19:18:28.740576 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735087 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735090 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735092 2579 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735095 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735098 2579 flags.go:64] FLAG: --lock-file="" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735101 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735103 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735106 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735112 2579 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735114 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735118 2579 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735121 2579 flags.go:64] FLAG: --logging-format="text" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735125 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735128 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735131 2579 flags.go:64] FLAG: --manifest-url="" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735134 2579 flags.go:64] FLAG: --manifest-url-header="" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735138 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735141 2579 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735146 2579 flags.go:64] FLAG: --max-pods="110" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735149 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735152 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735155 2579 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735158 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735161 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735164 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 19:18:28.741200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735166 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735174 2579 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735177 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735180 2579 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735183 2579 flags.go:64] FLAG: --pod-cidr="" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735186 2579 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735189 2579 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735192 2579 flags.go:64] FLAG: --pods-per-core="0" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735194 2579 flags.go:64] FLAG: --port="10250" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735197 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735200 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0451f66842405e3d7" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735203 2579 flags.go:64] FLAG: --qos-reserved="" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735206 2579 flags.go:64] FLAG: --read-only-port="10255" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735209 2579 flags.go:64] FLAG: --register-node="true" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735211 2579 flags.go:64] FLAG: --register-with-taints="" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735215 2579 flags.go:64] FLAG: --registry-burst="10" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735217 2579 flags.go:64] FLAG: --registry-qps="5" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735220 2579 flags.go:64] FLAG: --reserved-cpus="" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735223 2579 flags.go:64] FLAG: --reserved-memory="" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735228 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735231 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735234 2579 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735236 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735239 2579 flags.go:64] FLAG: --runonce="false" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735242 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735245 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735248 2579 flags.go:64] FLAG: --seccomp-default="false" Oct 30 19:18:28.741884 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735252 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735255 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735258 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735261 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735265 2579 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735268 2579 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735270 2579 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735273 2579 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735276 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735279 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735282 2579 flags.go:64] FLAG: --system-cgroups="" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735285 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735290 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735292 2579 flags.go:64] FLAG: --tls-cert-file="" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735295 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735300 2579 flags.go:64] FLAG: --tls-min-version="" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735302 2579 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735305 2579 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735308 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735311 2579 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735314 2579 flags.go:64] FLAG: --v="2" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735318 2579 flags.go:64] FLAG: --version="false" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735322 2579 flags.go:64] FLAG: --vmodule="" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735326 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735331 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 19:18:28.742651 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735467 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735471 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735474 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735477 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735479 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735482 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735485 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735489 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735492 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735494 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735497 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735501 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735505 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735508 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735511 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735513 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735516 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735519 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735522 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:18:28.743363 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735525 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735527 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735530 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735533 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735535 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735538 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735540 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735543 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735546 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735548 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735551 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735553 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735557 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735560 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735563 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735565 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735568 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735570 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735573 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735575 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:18:28.743911 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735579 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735582 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735584 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735587 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735590 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735592 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735595 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735597 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735600 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735602 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735605 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735607 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735610 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735612 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735615 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735618 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735620 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735623 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735625 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735628 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:18:28.744468 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735630 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735633 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735636 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735638 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735642 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735644 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735647 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735649 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735651 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735654 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735656 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735659 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735662 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735665 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735667 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735670 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735673 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735676 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735679 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735681 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:18:28.745222 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735684 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735686 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735689 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735691 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735693 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735696 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735711 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735714 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735717 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735719 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735722 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735724 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735727 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735729 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735732 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735734 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735738 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735740 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735744 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735747 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:18:28.745802 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735750 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735753 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735756 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735758 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735763 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735765 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735768 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.735796 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735832 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735837 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735841 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735844 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735847 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735849 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735852 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735855 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:18:28.746290 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735857 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735860 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735863 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735865 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735868 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735870 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735873 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735875 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735878 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735880 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735882 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735885 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735889 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735891 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735893 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735896 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735898 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735901 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735903 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735906 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:18:28.746684 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735908 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735912 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735915 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735917 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735920 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735922 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735925 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735928 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735931 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735933 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735935 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735938 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735940 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735943 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735945 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735948 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735950 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735952 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735955 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735957 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:18:28.747188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735960 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735963 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735965 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735968 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735972 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735975 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735977 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735980 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735983 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735985 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735988 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735990 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735993 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735996 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.735999 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736001 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736004 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736007 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736009 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:18:28.747667 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736012 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736015 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736017 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736019 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736022 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736025 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736027 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736031 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736035 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736038 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736040 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736043 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736045 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736048 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736050 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736052 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736055 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736059 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736061 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736063 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:18:28.748121 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736066 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736068 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736071 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736073 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736076 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736078 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736082 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736087 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736091 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736094 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736097 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736100 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736102 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736105 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736108 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736111 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736113 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736116 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.736118 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:18:28.748633 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.743552 2579 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.743569 2579 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.743588 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.743593 2579 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743663 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743668 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743671 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743674 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743677 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743680 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743682 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743685 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743688 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743690 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743693 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743696 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743713 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743718 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743721 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743724 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:18:28.749188 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743726 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743729 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743731 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743734 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743737 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743740 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743742 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743746 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743749 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743752 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743755 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743758 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743760 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743764 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743766 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743769 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743771 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743775 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743777 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743780 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:18:28.749654 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743782 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743785 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743787 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743790 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743792 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743795 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743798 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743800 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743803 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743806 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743808 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743811 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743815 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743820 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743824 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743827 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743830 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743832 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:18:28.750142 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743835 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743837 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743840 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743843 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743846 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743848 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743851 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743854 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743857 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743859 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743862 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743864 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743867 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743869 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743872 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743874 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743877 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743879 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743882 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:18:28.750562 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743885 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743887 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743890 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743892 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743894 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743897 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743899 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743902 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743904 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743906 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743909 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743911 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743914 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743916 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743919 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743922 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743925 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743928 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743931 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743933 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:18:28.751012 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743937 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743939 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743942 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743944 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743947 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743950 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743952 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743954 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743957 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743959 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743962 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743964 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.743967 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.743997 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744163 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744168 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:18:28.751520 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744171 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744174 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744176 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744179 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744182 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744184 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744187 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744189 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744192 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744194 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744196 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744200 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744202 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744208 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744212 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744215 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744217 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744220 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744222 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744225 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:18:28.751907 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744227 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744230 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744232 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744235 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744238 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744240 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744242 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744245 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744247 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744250 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744252 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744255 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744258 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744260 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744263 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744265 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744268 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744271 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744273 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744276 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:18:28.752371 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744278 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744281 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744283 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744286 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744289 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744291 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744294 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744297 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744299 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744302 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744305 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744307 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744310 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744312 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744315 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744317 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744320 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744323 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744325 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744328 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:18:28.752891 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744330 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744333 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744335 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744337 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744340 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744342 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744345 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744347 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744350 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744353 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744355 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744357 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744360 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744364 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744367 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744370 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744372 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744375 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744378 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:18:28.753617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744381 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744384 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744386 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744389 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744391 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744394 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744396 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744398 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744401 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744405 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744408 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744411 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744414 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744417 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744419 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744422 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744424 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744427 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744429 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:18:28.754265 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744432 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744434 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744437 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744439 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744441 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744444 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.744469 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744507 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744512 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744515 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744518 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744521 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744523 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744526 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744529 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744532 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:18:28.754741 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744535 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744537 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744540 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744542 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744545 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744547 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744549 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744553 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744555 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744557 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744560 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744563 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744565 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744568 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744571 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744573 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744576 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744578 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744581 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744583 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:18:28.755134 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744586 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744589 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744591 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744593 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744596 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744599 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744601 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744604 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744606 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744608 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744611 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744614 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744616 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744619 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744621 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744624 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744626 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744629 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744631 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744634 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:18:28.755617 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744637 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744639 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744642 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744644 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744647 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744649 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744651 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744654 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744656 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744659 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744661 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744664 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744666 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744669 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744672 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744675 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744677 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744681 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744684 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:18:28.756091 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744687 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744690 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744692 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744695 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744697 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744721 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744725 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744727 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744730 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744733 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744735 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744738 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744740 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744744 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744747 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744750 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744753 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744755 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744758 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:18:28.756539 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744760 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744763 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744765 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744767 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744770 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744772 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744779 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744782 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744784 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744787 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744789 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744792 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744795 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744797 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744800 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744805 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744807 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744810 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:28.744812 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:18:28.757021 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.744871 2579 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 19:18:28.757465 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.747060 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 19:18:28.757465 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.748118 2579 server.go:1014] "Starting client certificate rotation" Oct 30 19:18:28.757465 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.748215 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 19:18:28.757465 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.748976 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 19:18:28.757465 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.750284 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:18:28.757465 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.756065 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:18:28.771118 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.771098 2579 log.go:25] "Validated CRI v1 runtime API" Oct 30 19:18:28.774787 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.774766 2579 log.go:25] "Validated CRI v1 image API" Oct 30 19:18:28.775686 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.775659 2579 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 19:18:28.778216 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.778192 2579 fs.go:135] Filesystem UUIDs: map[3133ff10-e4fd-4d43-8770-a9b16ab6886a:/dev/nvme0n1p3 36e46223-66cd-4feb-8c0c-c99c05e7b95d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Oct 30 19:18:28.778283 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.778215 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 19:18:28.783461 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.783337 2579 manager.go:217] Machine: {Timestamp:2026-10-30 19:18:28.7818198 +0000 UTC m=+0.369019439 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499994 MemoryCapacity:32812003328 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27e24058190940ad438187542a2b68 SystemUUID:ec27e240-5819-0940-ad43-8187542a2b68 BootID:0def5452-661a-443c-ae13-8ba37d045f4a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16405999616 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ff:20:f1:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ff:20:f1:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:3e:85:e7:31:b2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812003328 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 19:18:28.783461 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.783458 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 19:18:28.783584 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.783539 2579 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 19:18:28.784550 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.784532 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 19:18:28.786868 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.786846 2579 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 19:18:28.787002 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.786870 2579 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-211.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 19:18:28.787043 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.787018 2579 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 19:18:28.787043 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.787026 2579 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 19:18:28.787103 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.787065 2579 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 19:18:28.787103 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.787091 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 19:18:28.788281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.788271 2579 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 19:18:28.793060 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.793047 2579 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 19:18:28.793170 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.793161 2579 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 19:18:28.793203 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.793196 2579 kubelet.go:493] "Attempting to sync node with API server" Oct 30 19:18:28.793233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.793212 2579 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 19:18:28.793233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.793221 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 19:18:28.793233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.793229 2579 kubelet.go:405] "Adding apiserver pod source" Oct 30 19:18:28.793331 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.793238 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 19:18:28.793612 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.793587 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 19:18:28.796140 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.796121 2579 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 19:18:28.803060 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.803044 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 19:18:28.804122 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804089 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s7t5z" Oct 30 19:18:28.804217 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804200 2579 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 19:18:28.804268 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804249 2579 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 19:18:28.804449 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804436 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 19:18:28.804494 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804458 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 19:18:28.804494 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804467 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 19:18:28.804494 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804479 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 19:18:28.804494 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804495 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 19:18:28.804654 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.804505 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 19:18:28.805198 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.805182 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 19:18:28.805198 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.805200 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 19:18:28.805312 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.805213 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 19:18:28.805312 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.805222 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 19:18:28.805312 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.805235 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 19:18:28.805312 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.805248 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 19:18:28.806110 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.806094 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 19:18:28.806243 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.806171 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 19:18:28.811110 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.811097 2579 server.go:1263] "Started kubelet" Oct 30 19:18:28.811258 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.811232 2579 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 19:18:28.811337 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.811221 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 19:18:28.811337 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.811292 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 19:18:28.811934 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.811916 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s7t5z" Oct 30 19:18:28.812069 ip-10-0-139-211 systemd[1]: Started Kubernetes Kubelet. Oct 30 19:18:28.812979 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.812921 2579 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 19:18:28.813427 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.813416 2579 server.go:317] "Adding debug handlers to kubelet server" Oct 30 19:18:28.816303 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.816283 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-211.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 19:18:28.816963 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.816931 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 19:18:28.816963 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.816934 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-211.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 19:18:28.817251 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.817218 2579 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 19:18:28.817685 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.816406 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-211.us-east-2.compute.internal.18e364cd2129f0f5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-211.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-211.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 19:18:28.811059445 +0000 UTC m=+0.398259083,LastTimestamp:2026-10-30 19:18:28.811059445 +0000 UTC m=+0.398259083,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-211.us-east-2.compute.internal,}" Oct 30 19:18:28.821026 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.821012 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 19:18:28.821026 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.821023 2579 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 19:18:28.821562 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.821208 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:28.821788 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.821774 2579 factory.go:55] Registering systemd factory Oct 30 19:18:28.821788 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.821789 2579 factory.go:223] Registration of the systemd container factory successfully Oct 30 19:18:28.821926 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.821815 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 19:18:28.821926 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.821917 2579 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 19:18:28.822025 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.821930 2579 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 19:18:28.822025 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.822023 2579 reconstruct.go:97] "Volume reconstruction finished" Oct 30 19:18:28.822114 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.822031 2579 reconciler.go:29] "Reconciler: start to sync state" Oct 30 19:18:28.822114 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.822051 2579 factory.go:153] Registering CRI-O factory Oct 30 19:18:28.822114 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.822063 2579 factory.go:223] Registration of the crio container factory successfully Oct 30 19:18:28.822114 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.822106 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 19:18:28.822218 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.822119 2579 factory.go:103] Registering Raw factory Oct 30 19:18:28.822218 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.822129 2579 manager.go:1199] Started watching for new ooms in manager Oct 30 19:18:28.823062 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.823047 2579 manager.go:319] Starting recovery of all containers Oct 30 19:18:28.835689 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.835538 2579 manager.go:324] Recovery completed Oct 30 19:18:28.836042 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.836023 2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:18:28.838563 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.838533 2579 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-211.us-east-2.compute.internal\" not found" node="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:28.841028 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.841015 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:18:28.843503 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.843482 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 19:18:28.843577 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.843521 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 19:18:28.843577 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.843533 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 30 19:18:28.844068 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.844054 2579 cpu_manager.go:226] "Starting" policy="none" Oct 30 19:18:28.844068 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.844067 2579 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 19:18:28.844181 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.844061 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:18:28.844181 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.844084 2579 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 19:18:28.844767 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.844752 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 19:18:28.844835 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.844781 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 19:18:28.844835 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.844796 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 30 19:18:28.846194 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.846180 2579 policy_none.go:50] "Start" Oct 30 19:18:28.846251 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.846196 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 19:18:28.846251 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.846209 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 19:18:28.848216 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.848205 2579 policy_none.go:44] "Start" Oct 30 19:18:28.894415 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.894374 2579 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 19:18:28.897939 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.897923 2579 manager.go:341] "Starting Device Plugin manager" Oct 30 19:18:28.898037 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.897951 2579 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 19:18:28.898037 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.897960 2579 server.go:87] "Starting device plugin registration server" Oct 30 19:18:28.898276 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.898261 2579 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 19:18:28.898341 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.898280 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 19:18:28.898415 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.898368 2579 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 19:18:28.898479 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.898458 2579 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 19:18:28.898479 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.898465 2579 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 19:18:28.899059 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.899039 2579 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 19:18:28.899143 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.899101 2579 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:28.967272 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.967245 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 19:18:28.969261 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.968626 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 19:18:28.969261 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.968648 2579 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 19:18:28.969261 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.968672 2579 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 19:18:28.969261 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:28.968765 2579 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 19:18:28.973792 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.973776 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:18:28.998843 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:28.998798 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:18:29.000130 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.000112 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 19:18:29.000216 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.000146 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 19:18:29.000216 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.000159 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 30 19:18:29.000216 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.000184 2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.009224 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.009210 2579 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.009273 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.009232 2579 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-211.us-east-2.compute.internal\": node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:29.023805 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.023778 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:29.068985 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.068959 2579 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal"] Oct 30 19:18:29.069069 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.069047 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:18:29.069866 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.069852 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 19:18:29.069929 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.069881 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 19:18:29.069929 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.069891 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 30 19:18:29.072154 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072142 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:18:29.072306 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.072344 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072326 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:18:29.072895 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072880 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 19:18:29.072895 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072892 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 19:18:29.072992 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072909 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 19:18:29.072992 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072913 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 19:18:29.072992 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072918 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 30 19:18:29.072992 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.072928 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 30 19:18:29.074949 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.074934 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.075017 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.074962 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:18:29.075654 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.075634 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 19:18:29.075745 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.075672 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 19:18:29.075745 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.075688 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 30 19:18:29.095034 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.095018 2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" node="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.097389 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.097373 2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" node="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.123016 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.122998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.123141 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.123023 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.123141 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.123040 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ce0953a30f3bb9b41a7629efbf90c90-config\") pod \"kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"1ce0953a30f3bb9b41a7629efbf90c90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.124049 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.124035 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:29.223453 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.223417 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.223453 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.223454 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.223658 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.223475 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ce0953a30f3bb9b41a7629efbf90c90-config\") pod \"kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"1ce0953a30f3bb9b41a7629efbf90c90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.223658 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.223504 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ce0953a30f3bb9b41a7629efbf90c90-config\") pod \"kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"1ce0953a30f3bb9b41a7629efbf90c90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.223658 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.223523 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.223658 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.223522 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.224482 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.224460 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:29.324680 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.324627 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:29.397827 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.397805 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.399955 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.399942 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.425420 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.425399 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:29.525899 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.525876 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:29.626301 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.626254 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-211.us-east-2.compute.internal\" not found" Oct 30 19:18:29.646851 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.646830 2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:18:29.721975 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.721949 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.733916 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.733896 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:18:29.735563 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.735547 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:18:29.743611 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.743599 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:18:29.747739 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.747727 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 19:18:29.747846 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.747824 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 19:18:29.747904 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.747887 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 19:18:29.747946 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.747890 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 19:18:29.779525 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.779498 2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:18:29.793572 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.793555 2579 apiserver.go:52] "Watching apiserver" Oct 30 19:18:29.800238 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.800215 2579 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 19:18:29.800909 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.800888 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-dvltm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls","openshift-cluster-node-tuning-operator/tuned-qfrrj","openshift-image-registry/node-ca-vvzd7","openshift-multus/multus-additional-cni-plugins-ccmmq","openshift-network-diagnostics/network-check-target-zv8fh","kube-system/global-pull-secret-syncer-f5lqj","openshift-dns/node-resolver-cgl7t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal","openshift-network-operator/iptables-alerter-m8hcp","kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal","openshift-multus/multus-95px9","openshift-multus/network-metrics-daemon-gvglc","kube-system/konnectivity-agent-6n7ll"] Oct 30 19:18:29.803267 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.803248 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.803348 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.803321 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:29.805188 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.805171 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:29.805276 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.805218 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:29.807735 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.807679 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.810345 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.810323 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.811629 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.811611 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 19:18:29.811723 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.811682 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:18:29.811941 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.811928 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rhqgv\"" Oct 30 19:18:29.812426 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.812414 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.814459 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.814444 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:29.814519 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.814502 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:29.814939 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.814926 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ctbj8\"" Oct 30 19:18:29.817181 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.817163 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:29.819401 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.819351 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:13:29 +0000 UTC" deadline="2028-03-31 13:13:15.726876841 +0000 UTC" Oct 30 19:18:29.819401 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.819402 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12425h54m45.90747922s" Oct 30 19:18:29.819512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.819455 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.821317 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.821300 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 19:18:29.821778 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.821764 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.824792 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.824774 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.825654 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825637 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:29.825752 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-slash\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.825752 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825676 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-netd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.825752 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-kubelet-config\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.825867 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-systemd\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.825867 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825808 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-env-overrides\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.825867 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825827 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovn-node-metrics-cert\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.825867 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-script-lib\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826014 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825874 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-run\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.826014 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825890 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-etc-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826014 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825912 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2hwl\" (UniqueName: \"kubernetes.io/projected/ac144cc0-bc7d-4430-a53b-194bcfb77145-kube-api-access-b2hwl\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.826014 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825933 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-kubernetes\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.826014 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825961 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw4qf\" (UniqueName: \"kubernetes.io/projected/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-kube-api-access-mw4qf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.826014 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.825979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-bin\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826245 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826063 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826245 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826124 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-system-cni-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.826245 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-ovn\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826245 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826204 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-var-lib-kubelet\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.826245 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:29.826472 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826274 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-log-socket\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826472 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826305 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826472 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826344 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-modprobe-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.826472 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826370 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-lib-modules\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.826472 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826391 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-systemd-units\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826472 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826415 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.826472 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826447 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.826472 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826471 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k59h\" (UniqueName: \"kubernetes.io/projected/a489b325-efa4-4d23-b7ae-978c813aa93e-kube-api-access-8k59h\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826532 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826562 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac144cc0-bc7d-4430-a53b-194bcfb77145-host-slash\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-tmp\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826625 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q796l\" (UniqueName: \"kubernetes.io/projected/2f59ba06-3da0-4638-9cd1-df7a19645b1d-kube-api-access-q796l\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-var-lib-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-node-log\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.826758 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826790 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-cnibin\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826821 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-registration-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826848 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-device-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysconfig\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826896 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-host\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-tuned\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826943 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rrcs\" (UniqueName: \"kubernetes.io/projected/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-kube-api-access-7rrcs\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.826969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w26kz\" (UniqueName: \"kubernetes.io/projected/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kube-api-access-w26kz\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827009 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac144cc0-bc7d-4430-a53b-194bcfb77145-iptables-alerter-script\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.827109 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827048 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-dbus\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827113 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9994\" (UniqueName: \"kubernetes.io/projected/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-kube-api-access-k9994\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827186 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827176 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-kubelet\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827333 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-socket-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827357 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-sys-fs\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827381 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-serviceca\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827403 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-host\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-config\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827501 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-conf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/756b72aa-da1a-4e51-8adf-df95022277d8-agent-certs\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:29.827600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827550 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/756b72aa-da1a-4e51-8adf-df95022277d8-konnectivity-ca\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:29.828208 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827605 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-netns\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.828208 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827644 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-systemd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.828208 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827672 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-etc-selinux\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.828208 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.827692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-sys\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.829460 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.829444 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-95px9" Oct 30 19:18:29.836998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.836976 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 19:18:29.857325 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.857304 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-skpcn" Oct 30 19:18:29.866014 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.865996 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-skpcn" Oct 30 19:18:29.912517 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.912489 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:18:29.912689 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.912625 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 19:18:29.912689 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.912654 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 19:18:29.912848 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.912794 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 19:18:29.912848 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.912816 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 19:18:29.912961 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.912894 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 19:18:29.912961 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.912928 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 19:18:29.912961 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.912946 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-dk8pw\"" Oct 30 19:18:29.913279 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913259 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 19:18:29.913331 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913283 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 19:18:29.913331 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913300 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 19:18:29.913331 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913321 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 19:18:29.913426 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913352 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-n5j86\"" Oct 30 19:18:29.913836 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913821 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 19:18:29.913883 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913838 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 19:18:29.913935 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913897 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 19:18:29.913935 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913928 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vj9wr\"" Oct 30 19:18:29.914046 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913939 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 19:18:29.914046 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913940 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 19:18:29.914046 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913899 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 19:18:29.914046 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913950 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vd8dh\"" Oct 30 19:18:29.914046 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913971 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 19:18:29.914046 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.913997 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 19:18:29.914299 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.914157 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vhzsf\"" Oct 30 19:18:29.914299 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.914159 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 19:18:29.914299 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.914167 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 19:18:29.914299 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.914182 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t67x5\"" Oct 30 19:18:29.914299 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.914187 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 19:18:29.914299 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.914198 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 19:18:29.914864 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.914849 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6rfl7\"" Oct 30 19:18:29.915018 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.915004 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 19:18:29.915018 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.915015 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 19:18:29.922887 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.922865 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.927906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysconfig\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.927948 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-host\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.927975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-tuned\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.927999 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rrcs\" (UniqueName: \"kubernetes.io/projected/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-kube-api-access-7rrcs\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w26kz\" (UniqueName: \"kubernetes.io/projected/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kube-api-access-w26kz\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928034 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac144cc0-bc7d-4430-a53b-194bcfb77145-iptables-alerter-script\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928050 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-dbus\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928073 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cnibin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928089 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9994\" (UniqueName: \"kubernetes.io/projected/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-kube-api-access-k9994\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928124 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-kubelet\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928140 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-socket-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928154 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-sys-fs\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-k8s-cni-cncf-io\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-multus-certs\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928200 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-serviceca\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.928627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928234 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-bin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928249 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-kubelet\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-host\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-config\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928297 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-conf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928320 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/756b72aa-da1a-4e51-8adf-df95022277d8-agent-certs\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/756b72aa-da1a-4e51-8adf-df95022277d8-konnectivity-ca\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928364 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eb0fad83-891c-4219-a590-846d4ca0089e-hosts-file\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928381 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-system-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-netns\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928458 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-netns\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928507 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-sys-fs\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928722 2579 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928751 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-socket-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928980 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-dbus\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.928987 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-serviceca\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929034 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac144cc0-bc7d-4430-a53b-194bcfb77145-iptables-alerter-script\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.929519 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929083 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-systemd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-etc-selinux\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-sys\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929161 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929186 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-slash\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929210 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-netd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929233 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysconfig\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-kubelet-config\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929285 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-kubelet-config\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929292 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-systemd\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929326 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-socket-dir-parent\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-env-overrides\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929380 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovn-node-metrics-cert\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-script-lib\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929431 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-run\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929456 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x74g6\" (UniqueName: \"kubernetes.io/projected/eb0fad83-891c-4219-a590-846d4ca0089e-kube-api-access-x74g6\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929482 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cni-binary-copy\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.930456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929503 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-etc-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929543 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-etc-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2hwl\" (UniqueName: \"kubernetes.io/projected/ac144cc0-bc7d-4430-a53b-194bcfb77145-kube-api-access-b2hwl\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929580 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-kubernetes\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw4qf\" (UniqueName: \"kubernetes.io/projected/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-kube-api-access-mw4qf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-bin\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929658 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-etc-kubernetes\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-system-cni-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929816 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-host\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929836 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-netd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929818 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929855 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-slash\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.929866 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929896 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-ovn\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929921 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-var-lib-kubelet\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931281 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.929940 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs podName:2f59ba06-3da0-4638-9cd1-df7a19645b1d nodeName:}" failed. No retries permitted until 2026-10-30 19:18:30.429920804 +0000 UTC m=+2.017120449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs") pod "network-metrics-daemon-gvglc" (UID: "2f59ba06-3da0-4638-9cd1-df7a19645b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929966 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-var-lib-kubelet\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929971 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.929999 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-log-socket\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930006 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-host\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930051 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-log-socket\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930060 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-modprobe-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930086 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-lib-modules\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb0fad83-891c-4219-a590-846d4ca0089e-tmp-dir\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930139 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-conf-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-systemd-units\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930208 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-kubernetes\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930219 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930245 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-daemon-config\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.931973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930288 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-env-overrides\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930348 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930400 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k59h\" (UniqueName: \"kubernetes.io/projected/a489b325-efa4-4d23-b7ae-978c813aa93e-kube-api-access-8k59h\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930410 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-etc-selinux\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac144cc0-bc7d-4430-a53b-194bcfb77145-host-slash\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930449 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-system-cni-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930480 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-tmp\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930495 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-ovn\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930509 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-multus\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930540 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q796l\" (UniqueName: \"kubernetes.io/projected/2f59ba06-3da0-4638-9cd1-df7a19645b1d-kube-api-access-q796l\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930566 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-var-lib-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930601 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-node-log\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-netns\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930659 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj9kx\" (UniqueName: \"kubernetes.io/projected/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-kube-api-access-qj9kx\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930686 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.933841 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930695 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-script-lib\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930757 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-hostroot\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930785 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-systemd\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930793 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/756b72aa-da1a-4e51-8adf-df95022277d8-konnectivity-ca\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930805 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-cnibin\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930843 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-registration-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930850 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-cnibin\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930879 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-modprobe-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-device-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930900 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-registration-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930904 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-kubelet\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930955 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-lib-modules\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930972 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-device-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930986 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930994 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-config\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931012 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.930976 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-systemd-units\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931093 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931107 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931131 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931214 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac144cc0-bc7d-4430-a53b-194bcfb77145-host-slash\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931316 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-sys\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931316 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-conf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931369 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-var-lib-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931391 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-systemd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931440 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-run\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.931442 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931469 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-bin\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.931490 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret podName:52f645a5-26c7-4cad-9834-71eacd7c9a88 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:30.431475868 +0000 UTC m=+2.018675513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret") pod "global-pull-secret-syncer-f5lqj" (UID: "52f645a5-26c7-4cad-9834-71eacd7c9a88") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931520 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.931546 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-node-log\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.932584 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-tuned\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.934998 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.934162 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-tmp\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.935587 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.935012 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovn-node-metrics-cert\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.935587 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.935042 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/756b72aa-da1a-4e51-8adf-df95022277d8-agent-certs\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:29.937028 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.936990 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:18:29.937028 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.937012 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:18:29.937028 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.937024 2579 projected.go:196] Error preparing data for projected volume kube-api-access-zz75l for pod openshift-network-diagnostics/network-check-target-zv8fh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:29.937164 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:29.937074 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l podName:d20919c9-e2a8-4e07-89ea-bc7cc726444c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:30.437058244 +0000 UTC m=+2.024257883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zz75l" (UniqueName: "kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l") pod "network-check-target-zv8fh" (UID: "d20919c9-e2a8-4e07-89ea-bc7cc726444c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:29.939776 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.939748 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w26kz\" (UniqueName: \"kubernetes.io/projected/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kube-api-access-w26kz\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:29.939927 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.939908 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2hwl\" (UniqueName: \"kubernetes.io/projected/ac144cc0-bc7d-4430-a53b-194bcfb77145-kube-api-access-b2hwl\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:29.939993 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.939929 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rrcs\" (UniqueName: \"kubernetes.io/projected/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-kube-api-access-7rrcs\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:29.943063 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.943039 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw4qf\" (UniqueName: \"kubernetes.io/projected/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-kube-api-access-mw4qf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:29.943966 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.943920 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q796l\" (UniqueName: \"kubernetes.io/projected/2f59ba06-3da0-4638-9cd1-df7a19645b1d-kube-api-access-q796l\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:29.944110 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.944093 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9994\" (UniqueName: \"kubernetes.io/projected/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-kube-api-access-k9994\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:29.944761 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:29.944743 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k59h\" (UniqueName: \"kubernetes.io/projected/a489b325-efa4-4d23-b7ae-978c813aa93e-kube-api-access-8k59h\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:30.010826 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.010790 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce0953a30f3bb9b41a7629efbf90c90.slice/crio-851adf46f4206e25d21c4eb5044a85e66cb1eb6fd6ed79875e712c311289df9a WatchSource:0}: Error finding container 851adf46f4206e25d21c4eb5044a85e66cb1eb6fd6ed79875e712c311289df9a: Status 404 returned error can't find the container with id 851adf46f4206e25d21c4eb5044a85e66cb1eb6fd6ed79875e712c311289df9a Oct 30 19:18:30.016943 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.016914 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:18:30.021028 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.020939 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2829c8c9b99a6ca60457b5349d0af7.slice/crio-7cb70437aa5333b50c7c30775d8ac6fffaefb70de87e55420f4848a0452f2027 WatchSource:0}: Error finding container 7cb70437aa5333b50c7c30775d8ac6fffaefb70de87e55420f4848a0452f2027: Status 404 returned error can't find the container with id 7cb70437aa5333b50c7c30775d8ac6fffaefb70de87e55420f4848a0452f2027 Oct 30 19:18:30.031434 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031414 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cnibin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-k8s-cni-cncf-io\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031458 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-multus-certs\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031473 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-bin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031487 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-kubelet\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031513 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eb0fad83-891c-4219-a590-846d4ca0089e-hosts-file\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031522 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cnibin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031522 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-multus-certs\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031546 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-bin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031569 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eb0fad83-891c-4219-a590-846d4ca0089e-hosts-file\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031571 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-k8s-cni-cncf-io\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031571 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-kubelet\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-system-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031648 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-system-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031650 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-socket-dir-parent\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031689 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x74g6\" (UniqueName: \"kubernetes.io/projected/eb0fad83-891c-4219-a590-846d4ca0089e-kube-api-access-x74g6\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:30.031725 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031694 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-socket-dir-parent\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031737 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cni-binary-copy\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031778 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-etc-kubernetes\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031816 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb0fad83-891c-4219-a590-846d4ca0089e-tmp-dir\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031834 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-etc-kubernetes\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031869 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-conf-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031893 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-daemon-config\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-multus\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031929 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031949 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-netns\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031957 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-multus\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031924 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-conf-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031990 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-netns\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.031994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj9kx\" (UniqueName: \"kubernetes.io/projected/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-kube-api-access-qj9kx\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.032037 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-hostroot\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.032094 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-hostroot\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032233 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.032124 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb0fad83-891c-4219-a590-846d4ca0089e-tmp-dir\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:30.032682 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.032246 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cni-binary-copy\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.032682 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.032336 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-daemon-config\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.039991 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.039973 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj9kx\" (UniqueName: \"kubernetes.io/projected/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-kube-api-access-qj9kx\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:18:30.040082 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.040009 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x74g6\" (UniqueName: \"kubernetes.io/projected/eb0fad83-891c-4219-a590-846d4ca0089e-kube-api-access-x74g6\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:30.132073 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.132048 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:18:30.141621 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.141598 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad2d631_46c4_42d5_8c8d_a0db93319a5e.slice/crio-5341462adf8fbe03cf1a3c0187c6c681f665b697c1c798b9964d74e773b8f331 WatchSource:0}: Error finding container 5341462adf8fbe03cf1a3c0187c6c681f665b697c1c798b9964d74e773b8f331: Status 404 returned error can't find the container with id 5341462adf8fbe03cf1a3c0187c6c681f665b697c1c798b9964d74e773b8f331 Oct 30 19:18:30.151735 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.151716 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:18:30.160794 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.160770 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda008f757_b4b8_48a7_bb30_ffa1c1d39b75.slice/crio-63021d1d189dc8b30d057e6df490079c78fb4aefac4c6ec31d5c47d3a3577f1d WatchSource:0}: Error finding container 63021d1d189dc8b30d057e6df490079c78fb4aefac4c6ec31d5c47d3a3577f1d: Status 404 returned error can't find the container with id 63021d1d189dc8b30d057e6df490079c78fb4aefac4c6ec31d5c47d3a3577f1d Oct 30 19:18:30.185920 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.185899 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:18:30.191945 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.191923 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:30.194856 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.194834 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda489b325_efa4_4d23_b7ae_978c813aa93e.slice/crio-dec3ac18faa447e3b545f7a27d0fb27fc7d7eae0d0bf55b8e7a25173514f3197 WatchSource:0}: Error finding container dec3ac18faa447e3b545f7a27d0fb27fc7d7eae0d0bf55b8e7a25173514f3197: Status 404 returned error can't find the container with id dec3ac18faa447e3b545f7a27d0fb27fc7d7eae0d0bf55b8e7a25173514f3197 Oct 30 19:18:30.200186 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.200165 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756b72aa_da1a_4e51_8adf_df95022277d8.slice/crio-8f1420f45ced8f5ba994a8774aad9eefbfc3b164d7d8e1a93f03787d3b430730 WatchSource:0}: Error finding container 8f1420f45ced8f5ba994a8774aad9eefbfc3b164d7d8e1a93f03787d3b430730: Status 404 returned error can't find the container with id 8f1420f45ced8f5ba994a8774aad9eefbfc3b164d7d8e1a93f03787d3b430730 Oct 30 19:18:30.212988 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.212968 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:30.218821 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.218640 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:18:30.224338 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.224314 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3d3354_c4de_4bf4_8d99_80808fe2e6f9.slice/crio-8603b15aa833f83b3e0ff835b3cd48635b80f2dbcfe9155e64f06ca04c6ce23f WatchSource:0}: Error finding container 8603b15aa833f83b3e0ff835b3cd48635b80f2dbcfe9155e64f06ca04c6ce23f: Status 404 returned error can't find the container with id 8603b15aa833f83b3e0ff835b3cd48635b80f2dbcfe9155e64f06ca04c6ce23f Oct 30 19:18:30.226947 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.226930 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:18:30.227228 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.227212 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc956bc7a_44f6_4d27_b9be_0d73f6dedd01.slice/crio-41bc887262dc5cdcb76c55ceae2cdf57d475c065a36aa4b1a413c425e0bbe4b4 WatchSource:0}: Error finding container 41bc887262dc5cdcb76c55ceae2cdf57d475c065a36aa4b1a413c425e0bbe4b4: Status 404 returned error can't find the container with id 41bc887262dc5cdcb76c55ceae2cdf57d475c065a36aa4b1a413c425e0bbe4b4 Oct 30 19:18:30.235464 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.235447 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac144cc0_bc7d_4430_a53b_194bcfb77145.slice/crio-09d2070eee1ace4e48896c774b152edd9c3b06af8ec157155f8d2819f80ff2be WatchSource:0}: Error finding container 09d2070eee1ace4e48896c774b152edd9c3b06af8ec157155f8d2819f80ff2be: Status 404 returned error can't find the container with id 09d2070eee1ace4e48896c774b152edd9c3b06af8ec157155f8d2819f80ff2be Oct 30 19:18:30.235515 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.235481 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:18:30.243921 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.243902 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb0fad83_891c_4219_a590_846d4ca0089e.slice/crio-082bf2a5b328f2f851966faa4fdc2865a5800fa728bc40e1d55b54cbbe585790 WatchSource:0}: Error finding container 082bf2a5b328f2f851966faa4fdc2865a5800fa728bc40e1d55b54cbbe585790: Status 404 returned error can't find the container with id 082bf2a5b328f2f851966faa4fdc2865a5800fa728bc40e1d55b54cbbe585790 Oct 30 19:18:30.260604 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.260587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-95px9" Oct 30 19:18:30.269276 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:18:30.269255 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcecdd80_d19c_4b8d_b38f_7067c98c7f98.slice/crio-19e0fc085bb97d28b7f06c6722ef3a37f3d8a9efc315cf798759c80ef2fc9709 WatchSource:0}: Error finding container 19e0fc085bb97d28b7f06c6722ef3a37f3d8a9efc315cf798759c80ef2fc9709: Status 404 returned error can't find the container with id 19e0fc085bb97d28b7f06c6722ef3a37f3d8a9efc315cf798759c80ef2fc9709 Oct 30 19:18:30.435525 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.435453 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:30.435525 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.435497 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:30.435716 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.435587 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:30.435716 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.435595 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:30.435716 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.435635 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs podName:2f59ba06-3da0-4638-9cd1-df7a19645b1d nodeName:}" failed. No retries permitted until 2026-10-30 19:18:31.4356236 +0000 UTC m=+3.022823224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs") pod "network-metrics-daemon-gvglc" (UID: "2f59ba06-3da0-4638-9cd1-df7a19645b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:30.435716 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.435648 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret podName:52f645a5-26c7-4cad-9834-71eacd7c9a88 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:31.435641897 +0000 UTC m=+3.022841522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret") pod "global-pull-secret-syncer-f5lqj" (UID: "52f645a5-26c7-4cad-9834-71eacd7c9a88") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:30.535985 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.535944 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:30.536161 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.536124 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:18:30.536161 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.536144 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:18:30.536161 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.536158 2579 projected.go:196] Error preparing data for projected volume kube-api-access-zz75l for pod openshift-network-diagnostics/network-check-target-zv8fh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:30.536345 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.536213 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l podName:d20919c9-e2a8-4e07-89ea-bc7cc726444c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:31.53619279 +0000 UTC m=+3.123392434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zz75l" (UniqueName: "kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l") pod "network-check-target-zv8fh" (UID: "d20919c9-e2a8-4e07-89ea-bc7cc726444c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:30.776994 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.776961 2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:18:30.873815 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.864387 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:18:30.873815 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.869692 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:13:30 +0000 UTC" deadline="2028-06-12 08:14:28.968707332 +0000 UTC" Oct 30 19:18:30.873815 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.869758 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14172h55m58.0989528s" Oct 30 19:18:30.973328 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.971620 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:30.973328 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:30.971752 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:30.986836 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.986762 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cgl7t" event={"ID":"eb0fad83-891c-4219-a590-846d4ca0089e","Type":"ContainerStarted","Data":"082bf2a5b328f2f851966faa4fdc2865a5800fa728bc40e1d55b54cbbe585790"} Oct 30 19:18:30.992360 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.992312 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m8hcp" event={"ID":"ac144cc0-bc7d-4430-a53b-194bcfb77145","Type":"ContainerStarted","Data":"09d2070eee1ace4e48896c774b152edd9c3b06af8ec157155f8d2819f80ff2be"} Oct 30 19:18:30.994052 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.994024 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6n7ll" event={"ID":"756b72aa-da1a-4e51-8adf-df95022277d8","Type":"ContainerStarted","Data":"8f1420f45ced8f5ba994a8774aad9eefbfc3b164d7d8e1a93f03787d3b430730"} Oct 30 19:18:30.999549 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:30.999463 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" event={"ID":"a489b325-efa4-4d23-b7ae-978c813aa93e","Type":"ContainerStarted","Data":"dec3ac18faa447e3b545f7a27d0fb27fc7d7eae0d0bf55b8e7a25173514f3197"} Oct 30 19:18:31.009702 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.009655 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vvzd7" event={"ID":"a008f757-b4b8-48a7-bb30-ffa1c1d39b75","Type":"ContainerStarted","Data":"63021d1d189dc8b30d057e6df490079c78fb4aefac4c6ec31d5c47d3a3577f1d"} Oct 30 19:18:31.011658 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.011630 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" event={"ID":"2ad2d631-46c4-42d5-8c8d-a0db93319a5e","Type":"ContainerStarted","Data":"5341462adf8fbe03cf1a3c0187c6c681f665b697c1c798b9964d74e773b8f331"} Oct 30 19:18:31.018159 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.018131 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"5c2829c8c9b99a6ca60457b5349d0af7","Type":"ContainerStarted","Data":"7cb70437aa5333b50c7c30775d8ac6fffaefb70de87e55420f4848a0452f2027"} Oct 30 19:18:31.028383 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.028316 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"1ce0953a30f3bb9b41a7629efbf90c90","Type":"ContainerStarted","Data":"851adf46f4206e25d21c4eb5044a85e66cb1eb6fd6ed79875e712c311289df9a"} Oct 30 19:18:31.038558 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.038483 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-95px9" event={"ID":"dcecdd80-d19c-4b8d-b38f-7067c98c7f98","Type":"ContainerStarted","Data":"19e0fc085bb97d28b7f06c6722ef3a37f3d8a9efc315cf798759c80ef2fc9709"} Oct 30 19:18:31.058327 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.058297 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" event={"ID":"c956bc7a-44f6-4d27-b9be-0d73f6dedd01","Type":"ContainerStarted","Data":"41bc887262dc5cdcb76c55ceae2cdf57d475c065a36aa4b1a413c425e0bbe4b4"} Oct 30 19:18:31.068715 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.068666 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"8603b15aa833f83b3e0ff835b3cd48635b80f2dbcfe9155e64f06ca04c6ce23f"} Oct 30 19:18:31.444561 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.444473 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:31.444561 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.444546 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:31.444867 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.444708 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:31.444867 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.444789 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs podName:2f59ba06-3da0-4638-9cd1-df7a19645b1d nodeName:}" failed. No retries permitted until 2026-10-30 19:18:33.444767779 +0000 UTC m=+5.031967421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs") pod "network-metrics-daemon-gvglc" (UID: "2f59ba06-3da0-4638-9cd1-df7a19645b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:31.445215 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.445194 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:31.445309 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.445255 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret podName:52f645a5-26c7-4cad-9834-71eacd7c9a88 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:33.445239746 +0000 UTC m=+5.032439392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret") pod "global-pull-secret-syncer-f5lqj" (UID: "52f645a5-26c7-4cad-9834-71eacd7c9a88") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:31.545568 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.545532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:31.545756 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.545697 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:18:31.545756 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.545713 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:18:31.545756 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.545751 2579 projected.go:196] Error preparing data for projected volume kube-api-access-zz75l for pod openshift-network-diagnostics/network-check-target-zv8fh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:31.545966 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.545815 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l podName:d20919c9-e2a8-4e07-89ea-bc7cc726444c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:33.545797589 +0000 UTC m=+5.132997220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zz75l" (UniqueName: "kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l") pod "network-check-target-zv8fh" (UID: "d20919c9-e2a8-4e07-89ea-bc7cc726444c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:31.870426 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.870378 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:13:30 +0000 UTC" deadline="2028-05-10 05:00:05.425016631 +0000 UTC" Oct 30 19:18:31.870426 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.870423 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13377h41m33.554598493s" Oct 30 19:18:31.969524 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.969487 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:31.969732 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.969647 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:31.970132 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:31.969966 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:31.970132 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:31.970076 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:32.274962 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:32.274671 2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:18:32.969967 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:32.969928 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:32.970470 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:32.970079 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:33.463188 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:33.463148 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:33.463455 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:33.463229 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:33.463455 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.463356 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:33.463455 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.463421 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret podName:52f645a5-26c7-4cad-9834-71eacd7c9a88 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:37.463403022 +0000 UTC m=+9.050602669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret") pod "global-pull-secret-syncer-f5lqj" (UID: "52f645a5-26c7-4cad-9834-71eacd7c9a88") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:33.463882 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.463861 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:33.463988 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.463925 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs podName:2f59ba06-3da0-4638-9cd1-df7a19645b1d nodeName:}" failed. No retries permitted until 2026-10-30 19:18:37.463907737 +0000 UTC m=+9.051107384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs") pod "network-metrics-daemon-gvglc" (UID: "2f59ba06-3da0-4638-9cd1-df7a19645b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:33.563705 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:33.563669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:33.563888 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.563867 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:18:33.563969 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.563897 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:18:33.563969 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.563910 2579 projected.go:196] Error preparing data for projected volume kube-api-access-zz75l for pod openshift-network-diagnostics/network-check-target-zv8fh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:33.564088 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.563972 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l podName:d20919c9-e2a8-4e07-89ea-bc7cc726444c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:37.563951154 +0000 UTC m=+9.151150797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zz75l" (UniqueName: "kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l") pod "network-check-target-zv8fh" (UID: "d20919c9-e2a8-4e07-89ea-bc7cc726444c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:33.973396 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:33.972666 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:33.973396 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.972846 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:33.973396 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:33.973233 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:33.973396 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:33.973343 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:34.971911 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:34.971878 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:34.972063 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:34.972031 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:35.106738 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.106448 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6n7ll" event={"ID":"756b72aa-da1a-4e51-8adf-df95022277d8","Type":"ContainerStarted","Data":"3db6ea9e74b2bd431420e8f1d2eab78745f9166727920973ad8802efcf336fca"} Oct 30 19:18:35.112656 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.111962 2579 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="df618cd317c940a46775c5871f63f5b8aedb298f1b94f43f9420b08634f062d7" exitCode=0 Oct 30 19:18:35.112656 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.112050 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" event={"ID":"a489b325-efa4-4d23-b7ae-978c813aa93e","Type":"ContainerDied","Data":"df618cd317c940a46775c5871f63f5b8aedb298f1b94f43f9420b08634f062d7"} Oct 30 19:18:35.120920 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.120204 2579 generic.go:356] "Generic (PLEG): container finished" podID="5c2829c8c9b99a6ca60457b5349d0af7" containerID="57540385dbe44a2ba598f090f4ff8930108a909c34ec3b54cd1202f5c5befdbf" exitCode=0 Oct 30 19:18:35.120920 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.120251 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"5c2829c8c9b99a6ca60457b5349d0af7","Type":"ContainerDied","Data":"57540385dbe44a2ba598f090f4ff8930108a909c34ec3b54cd1202f5c5befdbf"} Oct 30 19:18:35.139977 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.139935 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6n7ll" podStartSLOduration=1.452986881 podStartE2EDuration="6.139921506s" totalImagesPullingTime="4.686934625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:35.124916547 +0000 UTC m=+6.712116197" watchObservedRunningTime="2026-10-30 19:18:35.139921506 +0000 UTC m=+6.727121215" Oct 30 19:18:35.735599 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.735468 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:35.736606 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.736424 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:35.969989 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.969955 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:35.970164 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:35.970097 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:35.970586 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:35.970560 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:35.970729 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:35.970673 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:36.130034 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:36.129984 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"5c2829c8c9b99a6ca60457b5349d0af7","Type":"ContainerStarted","Data":"88606439f4f8da0cad1ab0ca31995f11d80d2e62b9dc395fe14765fe970f7def"} Oct 30 19:18:36.135080 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:36.134980 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"1ce0953a30f3bb9b41a7629efbf90c90","Type":"ContainerStarted","Data":"a3a6f783896cd569bfe79a82b0cc9525f8ce13c5ee34812f771d1956db5e2810"} Oct 30 19:18:36.176244 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:36.174921 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" podStartSLOduration=6.174902414 podStartE2EDuration="6.174902414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:36.154659923 +0000 UTC m=+7.741859574" watchObservedRunningTime="2026-10-30 19:18:36.174902414 +0000 UTC m=+7.762102065" Oct 30 19:18:36.971981 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:36.971938 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:36.972162 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:36.972052 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:37.144771 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.142787 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:18:37.144771 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.143874 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vvzd7" event={"ID":"a008f757-b4b8-48a7-bb30-ffa1c1d39b75","Type":"ContainerStarted","Data":"27711496952ecb455739e8eb42242174a368c8bb46ffc071d36ceff7394d0c3b"} Oct 30 19:18:37.161476 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.160428 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" podStartSLOduration=7.160412086 podStartE2EDuration="7.160412086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:36.176189485 +0000 UTC m=+7.763389138" watchObservedRunningTime="2026-10-30 19:18:37.160412086 +0000 UTC m=+8.747611737" Oct 30 19:18:37.161476 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.161065 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vvzd7" podStartSLOduration=2.083857969 podStartE2EDuration="8.161054155s" totalImagesPullingTime="6.077196186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:37.160255226 +0000 UTC m=+8.747454877" watchObservedRunningTime="2026-10-30 19:18:37.161054155 +0000 UTC m=+8.748253806" Oct 30 19:18:37.496270 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.496222 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:37.496470 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.496307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:37.496470 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.496448 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:37.496591 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.496512 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs podName:2f59ba06-3da0-4638-9cd1-df7a19645b1d nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.496498638 +0000 UTC m=+17.083698278 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs") pod "network-metrics-daemon-gvglc" (UID: "2f59ba06-3da0-4638-9cd1-df7a19645b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:37.496591 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.496585 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:37.496707 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.496613 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret podName:52f645a5-26c7-4cad-9834-71eacd7c9a88 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.496605438 +0000 UTC m=+17.083805080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret") pod "global-pull-secret-syncer-f5lqj" (UID: "52f645a5-26c7-4cad-9834-71eacd7c9a88") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:37.597174 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.597139 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:37.597345 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.597274 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:18:37.597345 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.597291 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:18:37.597345 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.597306 2579 projected.go:196] Error preparing data for projected volume kube-api-access-zz75l for pod openshift-network-diagnostics/network-check-target-zv8fh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:37.597526 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.597360 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l podName:d20919c9-e2a8-4e07-89ea-bc7cc726444c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.597348169 +0000 UTC m=+17.184547796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zz75l" (UniqueName: "kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l") pod "network-check-target-zv8fh" (UID: "d20919c9-e2a8-4e07-89ea-bc7cc726444c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:37.969662 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.968894 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:37.969662 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.969040 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:37.969662 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:37.969456 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:37.969662 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:37.969561 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:38.151552 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:38.151510 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" event={"ID":"c956bc7a-44f6-4d27-b9be-0d73f6dedd01","Type":"ContainerStarted","Data":"6e115e39d17e8d0a2e22fa014ba7b75232d2bd1a3a3c7558f51c9f04adec3bcf"} Oct 30 19:18:38.974057 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:38.972108 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:38.974057 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:38.972352 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:39.033518 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.032598 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:39.033518 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.032957 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:18:39.034276 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.034046 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:18:39.069447 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.069417 2579 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 19:18:39.155094 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.155017 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-95px9" event={"ID":"dcecdd80-d19c-4b8d-b38f-7067c98c7f98","Type":"ContainerStarted","Data":"15d8e6e8b46f11c3e850131d1473938563e8dbb420fcd8ef731890d4f848455d"} Oct 30 19:18:39.158393 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.158364 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" event={"ID":"c956bc7a-44f6-4d27-b9be-0d73f6dedd01","Type":"ContainerStarted","Data":"333144932f2e74dadc609bad0dad223ab952363adf132e5a9d4869224e045f7c"} Oct 30 19:18:39.173500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.173449 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-95px9" podStartSLOduration=1.604129779 podStartE2EDuration="10.173433185s" totalImagesPullingTime="8.569303406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:39.173065442 +0000 UTC m=+10.760265088" watchObservedRunningTime="2026-10-30 19:18:39.173433185 +0000 UTC m=+10.760632831" Oct 30 19:18:39.902994 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.902851 2579 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:18:39.069440254Z","UUID":"0caf8cb6-1dbb-4a9a-8d0f-d2e38200f5bd","Handler":null,"Name":"","Endpoint":""} Oct 30 19:18:39.904460 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.904437 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 19:18:39.904577 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.904467 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 19:18:39.969253 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.969219 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:39.969382 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:39.969347 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:39.969952 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:39.969794 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:39.969952 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:39.969908 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:40.163182 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:40.163115 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cgl7t" event={"ID":"eb0fad83-891c-4219-a590-846d4ca0089e","Type":"ContainerStarted","Data":"d9cae94203323ef188df4a60ba7c756f66bd2ef11225348a038988be9a4086f2"} Oct 30 19:18:40.165167 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:40.165133 2579 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="32147fe6a3dd373d3637067ad3648cb4715fe341cd8aab3244031ce070fa7165" exitCode=0 Oct 30 19:18:40.165456 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:40.165401 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" event={"ID":"a489b325-efa4-4d23-b7ae-978c813aa93e","Type":"ContainerDied","Data":"32147fe6a3dd373d3637067ad3648cb4715fe341cd8aab3244031ce070fa7165"} Oct 30 19:18:40.199869 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:40.199815 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-cgl7t" podStartSLOduration=2.244435719 podStartE2EDuration="11.199800066s" totalImagesPullingTime="8.955364347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:40.179068595 +0000 UTC m=+11.766268241" watchObservedRunningTime="2026-10-30 19:18:40.199800066 +0000 UTC m=+11.786999714" Oct 30 19:18:40.969954 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:40.969929 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:40.970086 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:40.970046 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:41.169828 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:41.169750 2579 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="341392f3a5a7d9875e75af507402fffafb26970fcc4b483ddf0acdf3e526ee7c" exitCode=0 Oct 30 19:18:41.170334 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:41.169848 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" event={"ID":"a489b325-efa4-4d23-b7ae-978c813aa93e","Type":"ContainerDied","Data":"341392f3a5a7d9875e75af507402fffafb26970fcc4b483ddf0acdf3e526ee7c"} Oct 30 19:18:41.173421 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:41.173043 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" event={"ID":"c956bc7a-44f6-4d27-b9be-0d73f6dedd01","Type":"ContainerStarted","Data":"a3ed2dcaea1dcaac7d2777dd673f2c2180850a5d0c9f8798fc8ee3c3c11ce571"} Oct 30 19:18:41.209049 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:41.208995 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" podStartSLOduration=2.730703534 podStartE2EDuration="12.208977356s" totalImagesPullingTime="9.478273822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:41.208411874 +0000 UTC m=+12.795611521" watchObservedRunningTime="2026-10-30 19:18:41.208977356 +0000 UTC m=+12.796177004" Oct 30 19:18:41.969188 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:41.969013 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:41.969321 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:41.969077 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:41.969321 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:41.969264 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:41.969392 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:41.969339 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:42.178217 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:42.178188 2579 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="09e98c52a1d3f4099a890487eaad0680f23bd33a1d8169bd0b1a83cfc73b555e" exitCode=0 Oct 30 19:18:42.178696 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:42.178272 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" event={"ID":"a489b325-efa4-4d23-b7ae-978c813aa93e","Type":"ContainerDied","Data":"09e98c52a1d3f4099a890487eaad0680f23bd33a1d8169bd0b1a83cfc73b555e"} Oct 30 19:18:42.179937 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:42.179910 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" event={"ID":"2ad2d631-46c4-42d5-8c8d-a0db93319a5e","Type":"ContainerStarted","Data":"22521fb7f1b9004766f2532087ac509db99b1983024fd3a652c2e2760c57daa1"} Oct 30 19:18:42.181637 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:42.181609 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m8hcp" event={"ID":"ac144cc0-bc7d-4430-a53b-194bcfb77145","Type":"ContainerStarted","Data":"74fb54fe4f9f4f4245d2fe4878ca196ff3675c66f861ffb8b877d62ec1df74b5"} Oct 30 19:18:42.215903 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:42.215859 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" podStartSLOduration=2.133960234 podStartE2EDuration="13.215846373s" totalImagesPullingTime="11.081886139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:42.215365104 +0000 UTC m=+13.802564752" watchObservedRunningTime="2026-10-30 19:18:42.215846373 +0000 UTC m=+13.803046020" Oct 30 19:18:42.229214 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:42.229136 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m8hcp" podStartSLOduration=3.872970659 podStartE2EDuration="13.22912096s" totalImagesPullingTime="9.356150301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:42.229017977 +0000 UTC m=+13.816217623" watchObservedRunningTime="2026-10-30 19:18:42.22912096 +0000 UTC m=+13.816320610" Oct 30 19:18:42.968940 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:42.968907 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:42.969105 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:42.969026 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:43.969735 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:43.969685 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:43.969735 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:43.969727 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:43.970259 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:43.969842 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:43.970259 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:43.969959 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:44.289966 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:44.289925 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:44.469013 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:44.468909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:44.969115 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:44.969082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:44.969323 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:44.969195 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:45.191161 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:45.191127 2579 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="1922bce96665d6593de528b72bcc862bffdc9f3cd612edc7b377c5f6624a0cef" exitCode=0 Oct 30 19:18:45.191633 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:45.191193 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" event={"ID":"a489b325-efa4-4d23-b7ae-978c813aa93e","Type":"ContainerDied","Data":"1922bce96665d6593de528b72bcc862bffdc9f3cd612edc7b377c5f6624a0cef"} Oct 30 19:18:45.555298 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:45.555223 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:45.555298 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:45.555290 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:45.555512 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.555394 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:45.555512 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.555394 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:45.555512 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.555454 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret podName:52f645a5-26c7-4cad-9834-71eacd7c9a88 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:01.555436775 +0000 UTC m=+33.142636406 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret") pod "global-pull-secret-syncer-f5lqj" (UID: "52f645a5-26c7-4cad-9834-71eacd7c9a88") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:18:45.555512 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.555471 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs podName:2f59ba06-3da0-4638-9cd1-df7a19645b1d nodeName:}" failed. No retries permitted until 2026-10-30 19:19:01.555462074 +0000 UTC m=+33.142661698 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs") pod "network-metrics-daemon-gvglc" (UID: "2f59ba06-3da0-4638-9cd1-df7a19645b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:18:45.655939 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:45.655913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:45.656078 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.656037 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:18:45.656078 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.656052 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:18:45.656078 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.656061 2579 projected.go:196] Error preparing data for projected volume kube-api-access-zz75l for pod openshift-network-diagnostics/network-check-target-zv8fh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:45.656169 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.656102 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l podName:d20919c9-e2a8-4e07-89ea-bc7cc726444c nodeName:}" failed. No retries permitted until 2026-10-30 19:19:01.656090329 +0000 UTC m=+33.243289968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zz75l" (UniqueName: "kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l") pod "network-check-target-zv8fh" (UID: "d20919c9-e2a8-4e07-89ea-bc7cc726444c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:18:45.957574 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:45.957405 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:45.969119 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:45.969099 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:45.969240 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.969221 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:45.969322 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:45.969256 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:45.969377 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:45.969344 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:46.065554 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.065528 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:46.196107 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.196074 2579 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="ddefcc5940c74de9e8013cf0ae0139098931de79de46364a774ce1926dcce4bc" exitCode=0 Oct 30 19:18:46.196603 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.196154 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" event={"ID":"a489b325-efa4-4d23-b7ae-978c813aa93e","Type":"ContainerDied","Data":"ddefcc5940c74de9e8013cf0ae0139098931de79de46364a774ce1926dcce4bc"} Oct 30 19:18:46.199001 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.198977 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:18:46.199426 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.199405 2579 generic.go:356] "Generic (PLEG): container finished" podID="4a3d3354-c4de-4bf4-8d99-80808fe2e6f9" containerID="5410b5ec2754c6e216d6d5da1985fcf37c1c08f0aea87cf0c6d06b2f8975ed90" exitCode=1 Oct 30 19:18:46.199509 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.199435 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"ca1c23eea6a83e89a4222a5d15f55c1e71b5120173c3fa03af10c77a4de9b85d"} Oct 30 19:18:46.199589 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.199575 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"9a260886db8513176adfbceeedfe648b8b5ef4ca80f9e93c7faf911e7cb9b606"} Oct 30 19:18:46.199642 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.199621 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"42757d6515a5f7a4a1b331dd2baeaec7588e496056b6c92874fae44caf2fc27d"} Oct 30 19:18:46.199715 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.199650 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"515df275458660e04f5d75e2a1c3801da5d05d2e4cc96d9ed113a48679a74843"} Oct 30 19:18:46.199715 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.199685 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerDied","Data":"5410b5ec2754c6e216d6d5da1985fcf37c1c08f0aea87cf0c6d06b2f8975ed90"} Oct 30 19:18:46.199831 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.199734 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"55ba293961b21a24a7fd68dbb1112bebe9d214f7f2d680d747f9d4d1965b46a5"} Oct 30 19:18:46.451939 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.451864 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:46.969136 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:46.969110 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:46.969285 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:46.969203 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:47.204427 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:47.204398 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" event={"ID":"a489b325-efa4-4d23-b7ae-978c813aa93e","Type":"ContainerStarted","Data":"f4e99a0e486e214f3fe0b683e5bc3bea233b0f933fa5de294f30aadf0d271ca5"} Oct 30 19:18:47.231497 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:47.231435 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ccmmq" podStartSLOduration=6.705645834 podStartE2EDuration="18.231424086s" totalImagesPullingTime="11.525778252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:47.230997672 +0000 UTC m=+18.818197522" watchObservedRunningTime="2026-10-30 19:18:47.231424086 +0000 UTC m=+18.818623733" Oct 30 19:18:47.969588 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:47.969561 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:47.969731 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:47.969655 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:47.969731 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:47.969680 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:47.969854 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:47.969758 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:48.210117 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:48.210098 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:18:48.210435 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:48.210416 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"31cbfdef80959af4a8f8c17c248e226b3aad3f5fa70462c2de69f0917502a2b1"} Oct 30 19:18:48.970182 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:48.970148 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:48.970359 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:48.970239 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:49.679458 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:49.679414 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:49.789805 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:49.789768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:49.955690 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:49.955606 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:49.969593 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:49.969573 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:49.969675 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:49.969664 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:49.969747 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:49.969731 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:49.969865 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:49.969846 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:50.081819 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:50.081789 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:50.190345 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:50.190316 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:50.294810 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:50.294602 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:50.969524 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:50.969497 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:50.969995 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:50.969607 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:50.984225 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:50.984191 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:51.219378 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.219358 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:18:51.219827 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.219782 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"266b310479c5c045fa48cf499134d50b92001868255163bcef84d381e5c0475a"} Oct 30 19:18:51.220045 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.220032 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:51.220110 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.220090 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:51.220220 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.220112 2579 scope.go:122] "RemoveContainer" containerID="5410b5ec2754c6e216d6d5da1985fcf37c1c08f0aea87cf0c6d06b2f8975ed90" Oct 30 19:18:51.237262 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.237245 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:51.782331 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.782300 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:51.957188 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.957121 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f5lqj"] Oct 30 19:18:51.957345 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.957297 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:51.957418 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:51.957394 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:51.963120 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.962989 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zv8fh"] Oct 30 19:18:51.963120 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.963105 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:51.963292 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:51.963186 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:51.963682 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.963661 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gvglc"] Oct 30 19:18:51.963800 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:51.963772 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:51.963887 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:51.963865 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:52.142491 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.142459 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:52.226251 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.226180 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:18:52.226600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.226577 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" event={"ID":"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9","Type":"ContainerStarted","Data":"25d87a6b392cf7afdd496aa9dd1955121a6a98e0cfb4e42756b906206fd014f5"} Oct 30 19:18:52.226873 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.226849 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:52.243445 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.243422 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:18:52.257526 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.257488 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" podStartSLOduration=7.722221077 podStartE2EDuration="23.25746839s" totalImagesPullingTime="15.535247313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:52.257158802 +0000 UTC m=+23.844358459" watchObservedRunningTime="2026-10-30 19:18:52.25746839 +0000 UTC m=+23.844668037" Oct 30 19:18:52.260892 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.260862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:52.383565 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.383527 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:52.526050 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:52.526018 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:53.558485 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:53.558444 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:53.764050 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:53.764022 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:53.961689 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:53.961605 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:53.969685 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:53.969665 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:53.969794 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:53.969738 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:53.969794 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:53.969749 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:53.969881 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:53.969832 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:53.969881 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:53.969866 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:53.969951 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:53.969925 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:54.180493 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:54.180457 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:54.365546 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:54.365491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:54.570641 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:54.570577 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cgl7t_eb0fad83-891c-4219-a590-846d4ca0089e/dns-node-resolver/0.log" Oct 30 19:18:54.748458 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:54.748417 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:55.324066 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.324032 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:55.465757 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.465725 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:55.586382 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.586296 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:55.772165 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.772138 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vvzd7_a008f757-b4b8-48a7-bb30-ffa1c1d39b75/node-ca/0.log" Oct 30 19:18:55.774757 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.774727 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:55.962410 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.962335 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:55.969538 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.969355 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:55.969627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.969410 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:55.969684 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:55.969410 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:55.969684 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:55.969647 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:55.969787 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:55.969714 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:55.969843 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:55.969816 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:56.185394 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:56.185360 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:56.370762 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:56.370735 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:56.482694 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:56.482661 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:57.046264 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.046231 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:57.159905 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.159868 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:57.546163 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.546128 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:57.699512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.699445 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:57.816295 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.816263 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:57.969055 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.968995 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:57.969157 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.969088 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:57.969157 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:57.969101 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:57.969157 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.969113 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:57.969252 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:57.969170 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:57.969252 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:57.969237 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:57.971927 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:57.971902 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:58.082922 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:58.082893 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:58.187200 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:58.187175 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:58.358102 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:58.358077 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:58.470370 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:58.470339 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:58.580796 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:58.580726 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:58.830282 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:58.830236 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:58.936263 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:58.936197 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:59.563446 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:59.563412 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:59.769956 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:59.769925 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:59.969446 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:59.969383 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:18:59.969446 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:59.969428 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:18:59.969596 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:59.969449 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:18:59.969596 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:59.969555 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:18:59.969661 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:59.969606 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:18:59.969695 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:18:59.969680 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:18:59.979952 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:18:59.979931 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:00.140102 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:00.140071 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:00.255746 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:00.255723 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:00.375268 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:00.375236 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:00.478306 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:00.478269 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:00.591940 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:00.591868 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:00.961145 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:00.961072 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:01.067185 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.067150 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:01.521549 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.521518 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:01.563437 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.563404 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:01.563595 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.563460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:01.563595 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.563542 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:19:01.563595 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.563550 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 19:19:01.563595 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.563593 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs podName:2f59ba06-3da0-4638-9cd1-df7a19645b1d nodeName:}" failed. No retries permitted until 2026-10-30 19:19:33.563579241 +0000 UTC m=+65.150778866 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs") pod "network-metrics-daemon-gvglc" (UID: "2f59ba06-3da0-4638-9cd1-df7a19645b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 19:19:01.563784 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.563608 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret podName:52f645a5-26c7-4cad-9834-71eacd7c9a88 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:33.563600741 +0000 UTC m=+65.150800365 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret") pod "global-pull-secret-syncer-f5lqj" (UID: "52f645a5-26c7-4cad-9834-71eacd7c9a88") : object "kube-system"/"original-pull-secret" not registered Oct 30 19:19:01.636621 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.636588 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:01.663937 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.663911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:01.664079 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.664027 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 19:19:01.664079 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.664040 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 19:19:01.664079 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.664049 2579 projected.go:196] Error preparing data for projected volume kube-api-access-zz75l for pod openshift-network-diagnostics/network-check-target-zv8fh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:19:01.664170 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.664089 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l podName:d20919c9-e2a8-4e07-89ea-bc7cc726444c nodeName:}" failed. No retries permitted until 2026-10-30 19:19:33.664077408 +0000 UTC m=+65.251277047 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zz75l" (UniqueName: "kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l") pod "network-check-target-zv8fh" (UID: "d20919c9-e2a8-4e07-89ea-bc7cc726444c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 19:19:01.768174 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.768146 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:01.969465 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.969399 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:01.969465 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.969426 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:01.969746 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.969494 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:19:01.969746 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.969537 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:01.969746 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.969609 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:19:01.969746 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:01.969648 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:19:01.981132 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:01.981106 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:02.085008 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:02.084973 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:02.194912 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:02.194889 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:02.362361 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:02.362328 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:03.166711 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:03.166678 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:03.362225 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:03.362201 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:03.579814 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:03.579782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:03.969180 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:03.969120 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:03.969180 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:03.969143 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:03.969347 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:03.969213 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:19:03.969347 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:03.969238 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:03.969347 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:03.969311 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:19:03.969458 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:03.969398 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:19:05.969597 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:05.969564 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:05.970013 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:05.969628 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:05.970013 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:05.969653 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:05.970013 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:05.969752 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:19:05.970013 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:05.969792 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:19:05.970013 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:05.969857 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:19:07.969847 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:07.969801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:07.969847 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:07.969846 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:07.970373 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:07.969912 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:07.970373 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:07.969931 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:19:07.970373 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:07.969989 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:19:07.970373 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:07.970039 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:19:09.969192 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:09.969153 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:09.969659 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:09.969239 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:09.969659 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:09.969327 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:09.969659 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:09.969349 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zv8fh" podUID="d20919c9-e2a8-4e07-89ea-bc7cc726444c" Oct 30 19:19:09.969659 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:09.969402 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5lqj" podUID="52f645a5-26c7-4cad-9834-71eacd7c9a88" Oct 30 19:19:09.969659 ip-10-0-139-211 kubenswrapper[2579]: E1030 19:19:09.969454 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvglc" podUID="2f59ba06-3da0-4638-9cd1-df7a19645b1d" Oct 30 19:19:10.149087 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.149053 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:10.312319 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.312284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:10.677460 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.677395 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeReady" Oct 30 19:19:10.677609 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.677500 2579 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 19:19:10.725830 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.725797 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lndpb"] Oct 30 19:19:10.736889 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.736867 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-79mtg"] Oct 30 19:19:10.737024 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.737009 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:10.761661 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.759282 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 19:19:10.761661 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.760051 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 19:19:10.761661 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.760136 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zxgfv\"" Oct 30 19:19:10.761661 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.760193 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 19:19:10.761935 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.761829 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lndpb"] Oct 30 19:19:10.761935 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.761868 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-79mtg"] Oct 30 19:19:10.762031 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.761957 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.762031 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.761999 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w9nxj"] Oct 30 19:19:10.772150 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.772132 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w9nxj"] Oct 30 19:19:10.772247 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.772215 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.838174 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838146 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-crio-socket\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.838284 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.838284 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838200 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bglv\" (UniqueName: \"kubernetes.io/projected/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-api-access-7bglv\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.838358 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838279 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5fac3e1-506c-4ce0-b992-9108addc1e81-cert\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:10.838358 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838322 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-metrics-tls\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.838423 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838369 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-config-volume\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.838423 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838389 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h77j\" (UniqueName: \"kubernetes.io/projected/a5fac3e1-506c-4ce0-b992-9108addc1e81-kube-api-access-5h77j\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:10.838423 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838404 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp899\" (UniqueName: \"kubernetes.io/projected/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-kube-api-access-mp899\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.838513 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838430 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-tmp-dir\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.838513 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838443 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-data-volume\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.838513 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.838457 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.855372 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.855351 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 19:19:10.855522 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.855504 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 19:19:10.855522 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.855514 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 19:19:10.855633 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.855515 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 19:19:10.855751 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.855738 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ktbkj\"" Oct 30 19:19:10.855805 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.855753 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 19:19:10.855805 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.855760 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 19:19:10.855805 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.855783 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gkcxf\"" Oct 30 19:19:10.939367 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.939367 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939341 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bglv\" (UniqueName: \"kubernetes.io/projected/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-api-access-7bglv\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.939512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5fac3e1-506c-4ce0-b992-9108addc1e81-cert\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:10.939512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-metrics-tls\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.939512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939409 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-config-volume\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.939512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939436 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h77j\" (UniqueName: \"kubernetes.io/projected/a5fac3e1-506c-4ce0-b992-9108addc1e81-kube-api-access-5h77j\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:10.939512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939459 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp899\" (UniqueName: \"kubernetes.io/projected/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-kube-api-access-mp899\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.939512 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939498 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-tmp-dir\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.939835 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939521 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-data-volume\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.939835 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.939835 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939585 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-crio-socket\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.939835 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939795 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-crio-socket\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.940035 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.939987 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-data-volume\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.940133 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.940114 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-tmp-dir\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.940191 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.940122 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-config-volume\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.940191 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.940181 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.943279 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.943251 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-metrics-tls\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:10.943361 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.943283 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.943408 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.943393 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5fac3e1-506c-4ce0-b992-9108addc1e81-cert\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:10.948289 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.948246 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bglv\" (UniqueName: \"kubernetes.io/projected/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-api-access-7bglv\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:10.949145 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.949121 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h77j\" (UniqueName: \"kubernetes.io/projected/a5fac3e1-506c-4ce0-b992-9108addc1e81-kube-api-access-5h77j\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:10.949235 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:10.949169 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp899\" (UniqueName: \"kubernetes.io/projected/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-kube-api-access-mp899\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:11.081176 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.081157 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:11.107364 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.107346 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:11.157940 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.157630 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:11.440180 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.440148 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lndpb"] Oct 30 19:19:11.441229 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.441202 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-79mtg"] Oct 30 19:19:11.443236 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:11.443213 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf1ec3f_be10_4166_ba73_8994e5d8d7a1.slice/crio-a7c1c487ad6573c94fff690a1770f0fd70a8193dc7f940347646eb30a49360f6 WatchSource:0}: Error finding container a7c1c487ad6573c94fff690a1770f0fd70a8193dc7f940347646eb30a49360f6: Status 404 returned error can't find the container with id a7c1c487ad6573c94fff690a1770f0fd70a8193dc7f940347646eb30a49360f6 Oct 30 19:19:11.444005 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:11.443982 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5fac3e1_506c_4ce0_b992_9108addc1e81.slice/crio-d2b19c163a18477bde18b361096e6b064aff151d42bdfa72301588ec6bcd0b4d WatchSource:0}: Error finding container d2b19c163a18477bde18b361096e6b064aff151d42bdfa72301588ec6bcd0b4d: Status 404 returned error can't find the container with id d2b19c163a18477bde18b361096e6b064aff151d42bdfa72301588ec6bcd0b4d Oct 30 19:19:11.446849 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.446828 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w9nxj"] Oct 30 19:19:11.449273 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:11.449256 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6afdc8a2_0be6_424f_ba13_99a3fcf436dc.slice/crio-19576ebd5b64807ef06b3b06de660200f5c0a6b6c03872353f82a6fc4bc5abcd WatchSource:0}: Error finding container 19576ebd5b64807ef06b3b06de660200f5c0a6b6c03872353f82a6fc4bc5abcd: Status 404 returned error can't find the container with id 19576ebd5b64807ef06b3b06de660200f5c0a6b6c03872353f82a6fc4bc5abcd Oct 30 19:19:11.969170 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.969135 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:11.969335 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.969193 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:11.969335 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.969307 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:11.973686 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.973656 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 19:19:11.974837 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.974694 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 19:19:11.974956 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.974928 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-t757m\"" Oct 30 19:19:11.974956 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.974934 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-4vgss\"" Oct 30 19:19:11.975065 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.974959 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 19:19:11.975065 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.974964 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs9gl\"" Oct 30 19:19:11.975065 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:11.975043 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 19:19:12.271753 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:12.271696 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79mtg" event={"ID":"acf1ec3f-be10-4166-ba73-8994e5d8d7a1","Type":"ContainerStarted","Data":"a7c1c487ad6573c94fff690a1770f0fd70a8193dc7f940347646eb30a49360f6"} Oct 30 19:19:12.274745 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:12.274688 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lndpb" event={"ID":"a5fac3e1-506c-4ce0-b992-9108addc1e81","Type":"ContainerStarted","Data":"d2b19c163a18477bde18b361096e6b064aff151d42bdfa72301588ec6bcd0b4d"} Oct 30 19:19:12.277074 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:12.277049 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w9nxj" event={"ID":"6afdc8a2-0be6-424f-ba13-99a3fcf436dc","Type":"ContainerStarted","Data":"59dbb4d6c695b9f5979ee5e48b87c3d9c63b1c729df3b1539cbbd0f42177e853"} Oct 30 19:19:12.277189 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:12.277107 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w9nxj" event={"ID":"6afdc8a2-0be6-424f-ba13-99a3fcf436dc","Type":"ContainerStarted","Data":"19576ebd5b64807ef06b3b06de660200f5c0a6b6c03872353f82a6fc4bc5abcd"} Oct 30 19:19:12.581043 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:12.581015 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cgl7t_eb0fad83-891c-4219-a590-846d4ca0089e/dns-node-resolver/0.log" Oct 30 19:19:13.283412 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.283384 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w9nxj" event={"ID":"6afdc8a2-0be6-424f-ba13-99a3fcf436dc","Type":"ContainerStarted","Data":"b6916dce37410b95930a72e4197e885adcfc036c8ec7b0b384c6f39f3dbc9739"} Oct 30 19:19:13.335981 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.335098 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c74d584d8-7k5wt"] Oct 30 19:19:13.337195 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.337173 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.341991 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.341948 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 19:19:13.344500 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.343501 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 19:19:13.347241 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.347178 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 19:19:13.348073 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.347960 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-87wls\"" Oct 30 19:19:13.348073 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.348045 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 19:19:13.348371 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.348345 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c74d584d8-7k5wt"] Oct 30 19:19:13.441211 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.441187 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 19:19:13.441301 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.441288 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 19:19:13.441367 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.441347 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 19:19:13.459731 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.459340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.459731 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.459406 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.459731 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.459460 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.459731 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.459506 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.459731 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.459540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.459731 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.459574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gtcg\" (UniqueName: \"kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.459731 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.459605 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.560694 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.560666 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.560800 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.560745 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.560800 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.560772 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gtcg\" (UniqueName: \"kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.560875 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.560815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.560875 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.560849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.560875 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.560862 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.560958 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.560890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.561555 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.561533 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.562530 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.562478 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.562530 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.562507 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.563373 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.563233 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.564559 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.564537 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.569178 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.569112 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.572426 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.572406 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gtcg\" (UniqueName: \"kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.708190 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.707913 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:13.981827 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.981803 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vvzd7_a008f757-b4b8-48a7-bb30-ffa1c1d39b75/node-ca/0.log" Oct 30 19:19:13.986211 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:13.986187 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c74d584d8-7k5wt"] Oct 30 19:19:13.989189 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:13.989161 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7e1781_2624_4b67_acbe_a36a05e238a9.slice/crio-450f86e9d510006611076f12c8ddb5613d56d965230923dcd308b88084990b17 WatchSource:0}: Error finding container 450f86e9d510006611076f12c8ddb5613d56d965230923dcd308b88084990b17: Status 404 returned error can't find the container with id 450f86e9d510006611076f12c8ddb5613d56d965230923dcd308b88084990b17 Oct 30 19:19:14.288573 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.288538 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w9nxj" event={"ID":"6afdc8a2-0be6-424f-ba13-99a3fcf436dc","Type":"ContainerStarted","Data":"247b5b0385fd7ebecaf23d68f62d183e168e6cb15ede8c28e763915f0a6baa2a"} Oct 30 19:19:14.290375 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.290348 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lndpb" event={"ID":"a5fac3e1-506c-4ce0-b992-9108addc1e81","Type":"ContainerStarted","Data":"837f9fd4dc7ac43bda147139398aca8d3a7f7d7455c3d98c2ab26079317b9f62"} Oct 30 19:19:14.291549 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.291526 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c74d584d8-7k5wt" event={"ID":"eb7e1781-2624-4b67-acbe-a36a05e238a9","Type":"ContainerStarted","Data":"450f86e9d510006611076f12c8ddb5613d56d965230923dcd308b88084990b17"} Oct 30 19:19:14.293464 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.293438 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79mtg" event={"ID":"acf1ec3f-be10-4166-ba73-8994e5d8d7a1","Type":"ContainerStarted","Data":"edddcb2dfff1933860009aa330d673db59e936256331583acdfa85ff4088cbbf"} Oct 30 19:19:14.293587 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.293499 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79mtg" event={"ID":"acf1ec3f-be10-4166-ba73-8994e5d8d7a1","Type":"ContainerStarted","Data":"232168e1e1a27fd4e733fcc81358a0b61cdbeca783ab186bb11f2529e445bd84"} Oct 30 19:19:14.293656 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.293644 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:14.309831 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.309788 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w9nxj" podStartSLOduration=1.991235517 podStartE2EDuration="4.309773761s" totalImagesPullingTime="2.318538244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:19:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:14.308054904 +0000 UTC m=+45.895254554" watchObservedRunningTime="2026-10-30 19:19:14.309773761 +0000 UTC m=+45.896973409" Oct 30 19:19:14.312600 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.312571 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:14.327110 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.327059 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lndpb" podStartSLOduration=1.549418961 podStartE2EDuration="4.32704246s" totalImagesPullingTime="2.777623499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:14.326012689 +0000 UTC m=+45.913212337" watchObservedRunningTime="2026-10-30 19:19:14.32704246 +0000 UTC m=+45.914242111" Oct 30 19:19:14.345722 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.345653 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-79mtg" podStartSLOduration=2.513778431 podStartE2EDuration="4.345635825s" totalImagesPullingTime="1.831857394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:14.345270245 +0000 UTC m=+45.932469894" watchObservedRunningTime="2026-10-30 19:19:14.345635825 +0000 UTC m=+45.932835473" Oct 30 19:19:14.380970 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.380942 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lndpb_a5fac3e1-506c-4ce0-b992-9108addc1e81/serve-healthcheck-canary/0.log" Oct 30 19:19:14.406693 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.406661 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8"] Oct 30 19:19:14.408377 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.408359 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.414933 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.414914 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 19:19:14.415084 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.414954 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 30 19:19:14.415084 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.415009 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 19:19:14.415084 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.415039 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-m4g9w\"" Oct 30 19:19:14.415084 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.415053 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 30 19:19:14.415501 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.415316 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 19:19:14.418834 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.418812 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8"] Oct 30 19:19:14.445193 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.445165 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:14.549932 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.549902 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:14.571537 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.571460 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.571747 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.571547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ef0160-1e62-4a92-8158-bdd45eebaeab-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.571747 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.571588 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7wtf\" (UniqueName: \"kubernetes.io/projected/14ef0160-1e62-4a92-8158-bdd45eebaeab-kube-api-access-f7wtf\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.571747 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.571621 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.672821 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.672791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.672943 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.672879 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ef0160-1e62-4a92-8158-bdd45eebaeab-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.672943 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.672926 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7wtf\" (UniqueName: \"kubernetes.io/projected/14ef0160-1e62-4a92-8158-bdd45eebaeab-kube-api-access-f7wtf\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.673197 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.672961 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.673852 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.673829 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ef0160-1e62-4a92-8158-bdd45eebaeab-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.675661 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.675638 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.675767 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.675732 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.683881 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.683862 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7wtf\" (UniqueName: \"kubernetes.io/projected/14ef0160-1e62-4a92-8158-bdd45eebaeab-kube-api-access-f7wtf\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.720971 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.720951 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:14.998636 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:14.998601 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8"] Oct 30 19:19:15.001463 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:15.001430 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14ef0160_1e62_4a92_8158_bdd45eebaeab.slice/crio-898246d70a75b8011907d3e92fc4c618c3cdb94f1a259505bb427c8522697b97 WatchSource:0}: Error finding container 898246d70a75b8011907d3e92fc4c618c3cdb94f1a259505bb427c8522697b97: Status 404 returned error can't find the container with id 898246d70a75b8011907d3e92fc4c618c3cdb94f1a259505bb427c8522697b97 Oct 30 19:19:15.298376 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.298307 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" event={"ID":"14ef0160-1e62-4a92-8158-bdd45eebaeab","Type":"ContainerStarted","Data":"898246d70a75b8011907d3e92fc4c618c3cdb94f1a259505bb427c8522697b97"} Oct 30 19:19:15.321762 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.321733 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c75bd5986-5gc79"] Oct 30 19:19:15.324379 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.324352 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.333778 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.333757 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 19:19:15.335446 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.335424 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c75bd5986-5gc79"] Oct 30 19:19:15.399377 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.399343 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:15.484350 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.484320 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.484517 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.484376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6hdf\" (UniqueName: \"kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.484517 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.484483 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.484627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.484517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.484627 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.484596 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.484742 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.484632 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.484742 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.484680 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.484742 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.484734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.530005 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.529975 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:15.585161 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585095 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.585161 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.585338 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585180 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6hdf\" (UniqueName: \"kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.585338 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.585338 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585222 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.585338 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.585338 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585294 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.585338 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585322 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.585951 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585928 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.586018 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.585990 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.586059 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.586040 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.586284 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.586268 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.586415 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.586392 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.588628 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.588609 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.589023 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.588998 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.622973 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.622952 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6hdf\" (UniqueName: \"kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.639906 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.639733 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:15.642944 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.642913 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:15.918569 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.918496 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c75bd5986-5gc79"] Oct 30 19:19:15.922103 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:15.922067 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4da4805_c7d7_408c_9f75_cd124fb938e9.slice/crio-a93f7693bcd7e5c9ee5a7d273f7618ea731544a6d881a66d04290362e78c90a2 WatchSource:0}: Error finding container a93f7693bcd7e5c9ee5a7d273f7618ea731544a6d881a66d04290362e78c90a2: Status 404 returned error can't find the container with id a93f7693bcd7e5c9ee5a7d273f7618ea731544a6d881a66d04290362e78c90a2 Oct 30 19:19:15.963474 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:15.963444 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:16.075065 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:16.075035 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:16.303111 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:16.303033 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c75bd5986-5gc79" event={"ID":"c4da4805-c7d7-408c-9f75-cd124fb938e9","Type":"ContainerStarted","Data":"a93f7693bcd7e5c9ee5a7d273f7618ea731544a6d881a66d04290362e78c90a2"} Oct 30 19:19:16.602473 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:16.602378 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:16.740187 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:16.740143 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:17.308453 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:17.308412 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c74d584d8-7k5wt" event={"ID":"eb7e1781-2624-4b67-acbe-a36a05e238a9","Type":"ContainerStarted","Data":"39189adffd3531844d3be29073a790933b6ea8c74856f1952f9d3de816c9c7d3"} Oct 30 19:19:17.311100 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:17.311070 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" event={"ID":"14ef0160-1e62-4a92-8158-bdd45eebaeab","Type":"ContainerStarted","Data":"279cecd04335641f5732b22bcecbe78616b48d40a21d1829652748a9a8e50229"} Oct 30 19:19:17.311351 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:17.311141 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" event={"ID":"14ef0160-1e62-4a92-8158-bdd45eebaeab","Type":"ContainerStarted","Data":"03bb14033d0e45d4eae69a3bcc3403770fe432847b84f3e8db593bf9aa16ba9a"} Oct 30 19:19:17.326173 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:17.326099 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7c74d584d8-7k5wt" podStartSLOduration=1.337540126 podStartE2EDuration="4.326081037s" totalImagesPullingTime="2.988540911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:17.325264502 +0000 UTC m=+48.912464150" watchObservedRunningTime="2026-10-30 19:19:17.326081037 +0000 UTC m=+48.913280687" Oct 30 19:19:17.343160 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:17.343111 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" podStartSLOduration=1.927378362 podStartE2EDuration="3.343098763s" totalImagesPullingTime="1.415720401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:17.342369982 +0000 UTC m=+48.929569642" watchObservedRunningTime="2026-10-30 19:19:17.343098763 +0000 UTC m=+48.930298410" Oct 30 19:19:18.315414 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.315377 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c75bd5986-5gc79" event={"ID":"c4da4805-c7d7-408c-9f75-cd124fb938e9","Type":"ContainerStarted","Data":"3a62ed38a6e0d3609ebc64798f42439e8a86887a408d63f49f961a3d6c8feefd"} Oct 30 19:19:18.566759 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.566643 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:18.710754 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.710693 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:18.814251 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.814207 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7c75bd5986-5gc79" podStartSLOduration=2.082910427 podStartE2EDuration="3.814192344s" totalImagesPullingTime="1.731281917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:18.336277221 +0000 UTC m=+49.923476865" watchObservedRunningTime="2026-10-30 19:19:18.814192344 +0000 UTC m=+50.401391991" Oct 30 19:19:18.814910 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.814895 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zjwlv"] Oct 30 19:19:18.816907 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.816871 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.821374 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.821356 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 19:19:18.821795 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.821775 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j9nbc\"" Oct 30 19:19:18.835360 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.835337 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:18.856444 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.856422 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 19:19:18.913803 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.913779 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-textfile\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.913803 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.913806 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spgqv\" (UniqueName: \"kubernetes.io/projected/3f35fd1e-89aa-430a-8198-703bc598c411-kube-api-access-spgqv\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.914013 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.913850 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.914013 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.913902 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.914013 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.913957 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-wtmp\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.914121 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.914090 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-tls\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.914168 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.914122 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-sys\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.914222 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.914185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-metrics-client-ca\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.914222 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.914207 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-root\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:18.922383 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.922360 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 19:19:18.950013 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:18.949990 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:19.015129 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015104 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-tls\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015225 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-sys\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015225 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015155 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-metrics-client-ca\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015225 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-root\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015332 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015229 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-textfile\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015332 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015232 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-sys\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015332 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015272 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spgqv\" (UniqueName: \"kubernetes.io/projected/3f35fd1e-89aa-430a-8198-703bc598c411-kube-api-access-spgqv\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015332 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015290 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-root\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015505 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015505 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015505 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015401 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-wtmp\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.015945 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.015847 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-wtmp\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.016253 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.016040 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-textfile\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.016334 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.016265 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-metrics-client-ca\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.018185 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.016411 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.020636 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.018608 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-tls\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.020636 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.018759 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.023580 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.023557 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spgqv\" (UniqueName: \"kubernetes.io/projected/3f35fd1e-89aa-430a-8198-703bc598c411-kube-api-access-spgqv\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.089013 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.088933 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:19.131682 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.131648 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:19.143761 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:19.143729 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f35fd1e_89aa_430a_8198_703bc598c411.slice/crio-adb804760b0bd20f9bab14ecb77f95d42c392048b3db06630ffb8f95222f9d68 WatchSource:0}: Error finding container adb804760b0bd20f9bab14ecb77f95d42c392048b3db06630ffb8f95222f9d68: Status 404 returned error can't find the container with id adb804760b0bd20f9bab14ecb77f95d42c392048b3db06630ffb8f95222f9d68 Oct 30 19:19:19.300581 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.300544 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:19.319735 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.319686 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjwlv" event={"ID":"3f35fd1e-89aa-430a-8198-703bc598c411","Type":"ContainerStarted","Data":"adb804760b0bd20f9bab14ecb77f95d42c392048b3db06630ffb8f95222f9d68"} Oct 30 19:19:19.414426 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.414348 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:19.529461 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.529423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:19.822992 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:19.822953 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:20.036461 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:20.036425 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:20.224591 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:20.224551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:20.323499 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:20.323472 2579 generic.go:356] "Generic (PLEG): container finished" podID="3f35fd1e-89aa-430a-8198-703bc598c411" containerID="f139de66cb6a0d9a35812bdd2da2577157a9cc1fc3787ddf32958404b15c2aa8" exitCode=0 Oct 30 19:19:20.323813 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:20.323534 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjwlv" event={"ID":"3f35fd1e-89aa-430a-8198-703bc598c411","Type":"ContainerDied","Data":"f139de66cb6a0d9a35812bdd2da2577157a9cc1fc3787ddf32958404b15c2aa8"} Oct 30 19:19:20.327504 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:20.327482 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:21.328737 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:21.328692 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjwlv" event={"ID":"3f35fd1e-89aa-430a-8198-703bc598c411","Type":"ContainerStarted","Data":"4d35a0c2f5993c2ebd724683719d69ae30138c95b0e6109ed2ec9a1101175707"} Oct 30 19:19:21.329097 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:21.328755 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjwlv" event={"ID":"3f35fd1e-89aa-430a-8198-703bc598c411","Type":"ContainerStarted","Data":"afb5d112b76c09619aa6af71252586041a895afc8c21c026253b2450296ebeb1"} Oct 30 19:19:21.348872 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:21.348830 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zjwlv" podStartSLOduration=2.293149701 podStartE2EDuration="3.348818191s" totalImagesPullingTime="980.470851ms" totalInitContainerRuntime="75.197639ms" isStatefulPod=true podCreationTimestamp="2026-10-30 19:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:21.347442179 +0000 UTC m=+52.934641826" watchObservedRunningTime="2026-10-30 19:19:21.348818191 +0000 UTC m=+52.936017838" Oct 30 19:19:21.785589 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:21.785554 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:21.890388 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:21.890359 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:21.998906 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:21.998873 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:22.679742 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:22.679687 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:22.791148 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:22.791117 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:23.623352 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:23.623313 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:23.708752 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:23.708719 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:23.709121 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:23.708778 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:23.713427 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:23.713406 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:23.722035 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:23.722013 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:24.242410 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:24.242385 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:24.300362 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:24.300341 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:24.343834 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:24.343812 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:25.092132 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:25.092087 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:25.640203 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:25.640169 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:25.640516 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:25.640235 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:25.647499 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:25.647477 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:26.181245 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.181205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:26.329979 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.329944 2579 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c75bd5986-5gc79"] Oct 30 19:19:26.336460 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.336434 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:26.349283 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.349262 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:26.359630 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.359610 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-54df886c98-kcrjn"] Oct 30 19:19:26.361692 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.361675 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.375694 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.375671 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54df886c98-kcrjn"] Oct 30 19:19:26.474247 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.474187 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.474247 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.474222 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.474391 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.474338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.474478 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.474402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ssdd\" (UniqueName: \"kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.474532 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.474514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.474589 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.474573 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.474728 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.474694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.474912 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.474892 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.575568 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.575532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ssdd\" (UniqueName: \"kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.575764 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.575574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.575764 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.575595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.575764 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.575750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.575917 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.575803 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.575917 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.575849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.575917 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.575875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.576056 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.575964 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.576056 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.576032 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.576478 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.576455 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.576739 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.576488 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.576739 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.576516 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.576739 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.576676 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.578442 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.578423 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.578551 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.578536 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.584641 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.584618 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ssdd\" (UniqueName: \"kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.673171 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.673143 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:26.946107 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:26.946076 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54df886c98-kcrjn"] Oct 30 19:19:26.948896 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:26.948859 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode646b5f0_ae93_4f27_a32f_78bbd45b837e.slice/crio-aa6056575f3baa76bffb23bb3f3e376a3186d92cb0f5d6814f8adaf61324c0f4 WatchSource:0}: Error finding container aa6056575f3baa76bffb23bb3f3e376a3186d92cb0f5d6814f8adaf61324c0f4: Status 404 returned error can't find the container with id aa6056575f3baa76bffb23bb3f3e376a3186d92cb0f5d6814f8adaf61324c0f4 Oct 30 19:19:27.350047 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:27.350006 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54df886c98-kcrjn" event={"ID":"e646b5f0-ae93-4f27-a32f-78bbd45b837e","Type":"ContainerStarted","Data":"551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485"} Oct 30 19:19:27.350463 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:27.350064 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54df886c98-kcrjn" event={"ID":"e646b5f0-ae93-4f27-a32f-78bbd45b837e","Type":"ContainerStarted","Data":"aa6056575f3baa76bffb23bb3f3e376a3186d92cb0f5d6814f8adaf61324c0f4"} Oct 30 19:19:27.369191 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:27.369149 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-54df886c98-kcrjn" podStartSLOduration=1.369134772 podStartE2EDuration="1.369134772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:27.368315829 +0000 UTC m=+58.955515475" watchObservedRunningTime="2026-10-30 19:19:27.369134772 +0000 UTC m=+58.956334411" Oct 30 19:19:27.376437 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:27.376402 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:28.555969 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:28.555934 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:28.676669 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:28.676629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:30.618250 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:30.618221 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:30.740876 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:30.740838 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:31.039833 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:31.039794 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:31.818208 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:31.818172 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:31.934312 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:31.934266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:32.556176 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:32.556146 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:32.660399 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:32.660372 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:33.630613 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.630583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:33.630994 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.630648 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:33.634664 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.634646 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 19:19:33.634726 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.634686 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 19:19:33.644134 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.644116 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:33.644216 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.644149 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:33.731243 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.731214 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:33.733886 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.733869 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 19:19:33.744182 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.744160 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 19:19:33.754901 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.754882 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:33.772145 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.772117 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:33.883138 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.883051 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:33.886714 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.886644 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs9gl\"" Oct 30 19:19:33.893981 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.893962 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:33.895188 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.895171 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-4vgss\"" Oct 30 19:19:33.900696 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.900679 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-t757m\"" Oct 30 19:19:33.903809 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.903786 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:33.909076 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:33.909058 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:34.179026 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:34.178946 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gvglc"] Oct 30 19:19:34.182467 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:34.182440 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f59ba06_3da0_4638_9cd1_df7a19645b1d.slice/crio-91e495142ac9377e15c7698fafe3fce5be5b40ac89717b2583489c22d6ab70b9 WatchSource:0}: Error finding container 91e495142ac9377e15c7698fafe3fce5be5b40ac89717b2583489c22d6ab70b9: Status 404 returned error can't find the container with id 91e495142ac9377e15c7698fafe3fce5be5b40ac89717b2583489c22d6ab70b9 Oct 30 19:19:34.219271 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:34.219244 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zv8fh"] Oct 30 19:19:34.224086 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:34.221657 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f5lqj"] Oct 30 19:19:34.226775 ip-10-0-139-211 kubenswrapper[2579]: W1030 19:19:34.226750 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f645a5_26c7_4cad_9834_71eacd7c9a88.slice/crio-9269626f9c5a62338464b96099da75433559d770a7a2244d87c7ca4d65938eba WatchSource:0}: Error finding container 9269626f9c5a62338464b96099da75433559d770a7a2244d87c7ca4d65938eba: Status 404 returned error can't find the container with id 9269626f9c5a62338464b96099da75433559d770a7a2244d87c7ca4d65938eba Oct 30 19:19:34.376334 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:34.376296 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f5lqj" event={"ID":"52f645a5-26c7-4cad-9834-71eacd7c9a88","Type":"ContainerStarted","Data":"9269626f9c5a62338464b96099da75433559d770a7a2244d87c7ca4d65938eba"} Oct 30 19:19:34.377523 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:34.377488 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zv8fh" event={"ID":"d20919c9-e2a8-4e07-89ea-bc7cc726444c","Type":"ContainerStarted","Data":"55853780a57810149cdbd60dca7709382a1b6235d6a62639cecbb936f95aff1f"} Oct 30 19:19:34.380161 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:34.378861 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvglc" event={"ID":"2f59ba06-3da0-4638-9cd1-df7a19645b1d","Type":"ContainerStarted","Data":"91e495142ac9377e15c7698fafe3fce5be5b40ac89717b2583489c22d6ab70b9"} Oct 30 19:19:36.187172 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:36.187128 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:36.289922 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:36.289886 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:36.390853 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:36.390813 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvglc" event={"ID":"2f59ba06-3da0-4638-9cd1-df7a19645b1d","Type":"ContainerStarted","Data":"2d565a9f4cf72db523ebf7e8a03caf11d8dda0816932d8afc38d8340fc867320"} Oct 30 19:19:36.391022 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:36.390884 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvglc" event={"ID":"2f59ba06-3da0-4638-9cd1-df7a19645b1d","Type":"ContainerStarted","Data":"4cab673ba8fc9f0f5e0e0a5aaa0843db472166eae540959d38bf6a135107b645"} Oct 30 19:19:36.408953 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:36.407833 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gvglc" podStartSLOduration=66.149839435 podStartE2EDuration="1m7.407812197s" totalImagesPullingTime="1.257972762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:36.406765928 +0000 UTC m=+67.993965766" watchObservedRunningTime="2026-10-30 19:19:36.407812197 +0000 UTC m=+67.995011845" Oct 30 19:19:36.673750 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:36.673687 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:36.673938 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:36.673780 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:36.679695 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:36.679667 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:37.401390 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:37.401359 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:37.446156 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:37.446125 2579 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c74d584d8-7k5wt"] Oct 30 19:19:38.400498 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:38.400455 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zv8fh" event={"ID":"d20919c9-e2a8-4e07-89ea-bc7cc726444c","Type":"ContainerStarted","Data":"e690c3491e0ea130b441680a22130f05bb37044433ea8abeac80031467c94c1a"} Oct 30 19:19:38.419195 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:38.419145 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zv8fh" podStartSLOduration=66.050544483 podStartE2EDuration="1m9.419130219s" totalImagesPullingTime="3.368585736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:38.417851777 +0000 UTC m=+70.005051417" watchObservedRunningTime="2026-10-30 19:19:38.419130219 +0000 UTC m=+70.006329865" Oct 30 19:19:39.214886 ip-10-0-139-211 kubenswrapper[2579]: I1030 19:19:39.214757 2579 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:19:39.214932 ip-10-0-139-211 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 19:19:39.224293 ip-10-0-139-211 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 19:19:39.224457 ip-10-0-139-211 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 19:19:39.224505 ip-10-0-139-211 systemd[1]: kubelet.service: Consumed 3.809s CPU time. Oct 30 19:19:39.240253 ip-10-0-139-211 systemd[1]: Starting Kubernetes Kubelet... Oct 30 19:19:39.304913 ip-10-0-139-211 kubenswrapper[5627]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:19:39.304913 ip-10-0-139-211 kubenswrapper[5627]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 19:19:39.304913 ip-10-0-139-211 kubenswrapper[5627]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:19:39.304913 ip-10-0-139-211 kubenswrapper[5627]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 19:19:39.307092 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307005 5627 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:19:39.307092 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307091 5627 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:19:39.307092 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307096 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307100 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307103 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307106 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307109 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307111 5627 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307114 5627 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307117 5627 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307120 5627 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307122 5627 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307125 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307127 5627 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307129 5627 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307132 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307135 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307137 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307140 5627 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307142 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307145 5627 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:19:39.307186 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307148 5627 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307151 5627 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307153 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307156 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307159 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307167 5627 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307171 5627 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307176 5627 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307180 5627 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307182 5627 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307185 5627 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307189 5627 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307194 5627 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307197 5627 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307200 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307204 5627 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307206 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307209 5627 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307212 5627 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:19:39.307627 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307214 5627 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307217 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307219 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307222 5627 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307224 5627 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307227 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307230 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307233 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307236 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307238 5627 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307241 5627 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307243 5627 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307246 5627 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307249 5627 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307252 5627 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307255 5627 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307257 5627 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307261 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307264 5627 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307267 5627 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:19:39.308102 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307269 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307272 5627 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307275 5627 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307277 5627 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307280 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307282 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307285 5627 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307288 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307290 5627 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307292 5627 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307295 5627 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307297 5627 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307300 5627 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307303 5627 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307305 5627 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307307 5627 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307310 5627 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307318 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307320 5627 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307323 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:19:39.308574 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307326 5627 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307328 5627 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307330 5627 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307333 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307337 5627 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307339 5627 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307342 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307344 5627 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307347 5627 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307350 5627 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307352 5627 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307354 5627 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307359 5627 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307361 5627 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307364 5627 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307366 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307369 5627 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307371 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307375 5627 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307377 5627 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:19:39.309075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307380 5627 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307382 5627 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307385 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307387 5627 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307390 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.307392 5627 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308183 5627 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308190 5627 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308193 5627 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308199 5627 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308202 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308204 5627 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308207 5627 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308210 5627 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308212 5627 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308215 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308218 5627 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308221 5627 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308223 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:19:39.309563 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308226 5627 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308228 5627 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308231 5627 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308233 5627 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308236 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308238 5627 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308241 5627 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308244 5627 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308247 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308249 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308252 5627 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308254 5627 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308257 5627 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308259 5627 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308261 5627 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308264 5627 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308266 5627 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308269 5627 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308271 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308273 5627 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:19:39.310054 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308276 5627 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308279 5627 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308283 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308286 5627 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308288 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308291 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308293 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308296 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308298 5627 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308301 5627 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308304 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308306 5627 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308309 5627 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308311 5627 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308314 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308317 5627 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308319 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308321 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308324 5627 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:19:39.310534 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308327 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308329 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308332 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308334 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308337 5627 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308339 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308341 5627 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308344 5627 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308347 5627 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308349 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308352 5627 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308354 5627 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308356 5627 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308359 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308362 5627 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308366 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308368 5627 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308371 5627 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308374 5627 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308376 5627 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:19:39.311035 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308379 5627 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308381 5627 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308384 5627 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308386 5627 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308389 5627 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308391 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308394 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308396 5627 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308400 5627 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308405 5627 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308407 5627 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308410 5627 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308413 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308416 5627 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308418 5627 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308421 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308423 5627 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308426 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308428 5627 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:19:39.311535 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308432 5627 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308435 5627 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308438 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308441 5627 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308443 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308447 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308449 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308453 5627 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308456 5627 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308459 5627 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308461 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308464 5627 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308466 5627 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308469 5627 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308471 5627 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308989 5627 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308996 5627 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.308999 5627 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309002 5627 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309004 5627 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:19:39.312007 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309007 5627 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309009 5627 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309012 5627 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309015 5627 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309017 5627 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309020 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309023 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309026 5627 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309029 5627 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309031 5627 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309034 5627 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309036 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309039 5627 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309041 5627 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309044 5627 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309046 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309049 5627 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309052 5627 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309055 5627 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309058 5627 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:19:39.312506 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309062 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309065 5627 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309068 5627 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309071 5627 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309073 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309076 5627 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309079 5627 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309082 5627 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309085 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309087 5627 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309090 5627 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309092 5627 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309095 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309098 5627 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309100 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309103 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309105 5627 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309107 5627 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309110 5627 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:19:39.313013 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309113 5627 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309115 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309117 5627 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309120 5627 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309122 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309125 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309127 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309129 5627 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309132 5627 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309134 5627 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309137 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309139 5627 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309142 5627 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309146 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309149 5627 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309152 5627 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309154 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309157 5627 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309160 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:19:39.313468 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309162 5627 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309165 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309167 5627 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309169 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309172 5627 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309174 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309177 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309179 5627 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309182 5627 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309184 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309187 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309189 5627 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309192 5627 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309194 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309197 5627 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309199 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309202 5627 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309205 5627 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309207 5627 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:19:39.313998 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309209 5627 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309212 5627 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309214 5627 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309217 5627 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309219 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309222 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309225 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309228 5627 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309230 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309233 5627 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309235 5627 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309237 5627 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309240 5627 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309243 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309245 5627 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309248 5627 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309250 5627 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309252 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309255 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309257 5627 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:19:39.314439 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309260 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309262 5627 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309266 5627 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.309270 5627 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309366 5627 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309373 5627 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309380 5627 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309384 5627 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309389 5627 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309392 5627 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309397 5627 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309401 5627 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309404 5627 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309407 5627 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309410 5627 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309414 5627 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309417 5627 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309420 5627 flags.go:64] FLAG: --cgroup-root="" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309423 5627 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309426 5627 flags.go:64] FLAG: --client-ca-file="" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309429 5627 flags.go:64] FLAG: --cloud-provider="external" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309432 5627 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309436 5627 flags.go:64] FLAG: --cluster-domain="" Oct 30 19:19:39.314942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309438 5627 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309442 5627 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309444 5627 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309448 5627 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309452 5627 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309457 5627 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309460 5627 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309463 5627 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309466 5627 flags.go:64] FLAG: --contention-profiling="false" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309469 5627 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309472 5627 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309475 5627 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309478 5627 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309483 5627 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309486 5627 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309489 5627 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309492 5627 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309495 5627 flags.go:64] FLAG: --enable-server="true" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309498 5627 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309503 5627 flags.go:64] FLAG: --event-burst="100" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309506 5627 flags.go:64] FLAG: --event-qps="50" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309509 5627 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309512 5627 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309515 5627 flags.go:64] FLAG: --eviction-hard="" Oct 30 19:19:39.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309519 5627 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309522 5627 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309536 5627 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309540 5627 flags.go:64] FLAG: --eviction-soft="" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309543 5627 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309546 5627 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309549 5627 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309551 5627 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309554 5627 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309557 5627 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309560 5627 flags.go:64] FLAG: --feature-gates="" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309564 5627 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309567 5627 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309572 5627 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309575 5627 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309578 5627 flags.go:64] FLAG: --healthz-port="10248" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309581 5627 flags.go:64] FLAG: --help="false" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309584 5627 flags.go:64] FLAG: --hostname-override="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309587 5627 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309590 5627 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309593 5627 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309596 5627 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309599 5627 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 19:19:39.316506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309602 5627 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309605 5627 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309609 5627 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309612 5627 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309615 5627 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309618 5627 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309621 5627 flags.go:64] FLAG: --kube-reserved="" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309624 5627 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309626 5627 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309630 5627 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309633 5627 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309636 5627 flags.go:64] FLAG: --lock-file="" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309638 5627 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309641 5627 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309644 5627 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309649 5627 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309652 5627 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309655 5627 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309658 5627 flags.go:64] FLAG: --logging-format="text" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309660 5627 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309663 5627 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309666 5627 flags.go:64] FLAG: --manifest-url="" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309671 5627 flags.go:64] FLAG: --manifest-url-header="" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309675 5627 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309679 5627 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 19:19:39.317074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309683 5627 flags.go:64] FLAG: --max-pods="110" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309686 5627 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309689 5627 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309691 5627 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309694 5627 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309697 5627 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309717 5627 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309721 5627 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309728 5627 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309731 5627 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309734 5627 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309737 5627 flags.go:64] FLAG: --pod-cidr="" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309741 5627 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309744 5627 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309749 5627 flags.go:64] FLAG: --pods-per-core="0" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309752 5627 flags.go:64] FLAG: --port="10250" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309755 5627 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309757 5627 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0451f66842405e3d7" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309761 5627 flags.go:64] FLAG: --qos-reserved="" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309764 5627 flags.go:64] FLAG: --read-only-port="10255" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309766 5627 flags.go:64] FLAG: --register-node="true" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309769 5627 flags.go:64] FLAG: --register-with-taints="" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309773 5627 flags.go:64] FLAG: --registry-burst="10" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309776 5627 flags.go:64] FLAG: --registry-qps="5" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309779 5627 flags.go:64] FLAG: --reserved-cpus="" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309782 5627 flags.go:64] FLAG: --reserved-memory="" Oct 30 19:19:39.317682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309785 5627 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309788 5627 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309791 5627 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309796 5627 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309799 5627 flags.go:64] FLAG: --runonce="false" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309802 5627 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309805 5627 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309808 5627 flags.go:64] FLAG: --seccomp-default="false" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309811 5627 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309813 5627 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309816 5627 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309820 5627 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309823 5627 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309825 5627 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309828 5627 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309831 5627 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309834 5627 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309837 5627 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309840 5627 flags.go:64] FLAG: --system-cgroups="" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309844 5627 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309851 5627 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309854 5627 flags.go:64] FLAG: --tls-cert-file="" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309856 5627 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309860 5627 flags.go:64] FLAG: --tls-min-version="" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309863 5627 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 19:19:39.318313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309866 5627 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309869 5627 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309872 5627 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309875 5627 flags.go:64] FLAG: --v="2" Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309879 5627 flags.go:64] FLAG: --version="false" Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309883 5627 flags.go:64] FLAG: --vmodule="" Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309887 5627 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.309890 5627 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310024 5627 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310028 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310033 5627 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310036 5627 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310038 5627 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310041 5627 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310043 5627 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310046 5627 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310049 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310051 5627 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310053 5627 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310056 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310059 5627 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:19:39.318918 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310061 5627 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310064 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310066 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310069 5627 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310071 5627 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310074 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310077 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310080 5627 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310083 5627 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310085 5627 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310088 5627 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310090 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310093 5627 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310096 5627 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310098 5627 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310101 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310103 5627 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310106 5627 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310108 5627 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310111 5627 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:19:39.319581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310113 5627 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310117 5627 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310119 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310122 5627 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310125 5627 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310127 5627 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310130 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310132 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310135 5627 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310138 5627 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310142 5627 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310145 5627 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310148 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310150 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310153 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310155 5627 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310158 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310160 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310163 5627 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:19:39.320101 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310165 5627 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310168 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310171 5627 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310173 5627 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310175 5627 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310178 5627 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310181 5627 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310183 5627 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310186 5627 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310188 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310190 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310208 5627 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310213 5627 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310215 5627 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310220 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310222 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310225 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310228 5627 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310231 5627 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310233 5627 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:19:39.320561 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310236 5627 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310238 5627 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310241 5627 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310243 5627 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310246 5627 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310248 5627 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310251 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310253 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310256 5627 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310259 5627 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310261 5627 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310264 5627 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310266 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310269 5627 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310272 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310274 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310277 5627 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310279 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310282 5627 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:19:39.321096 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310284 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310287 5627 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310289 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310291 5627 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310294 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310296 5627 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310299 5627 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310302 5627 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310305 5627 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310308 5627 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310310 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310312 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310316 5627 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310320 5627 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310324 5627 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:19:39.321620 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.310348 5627 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310388 5627 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310394 5627 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310398 5627 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310401 5627 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310404 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310407 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310409 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310412 5627 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310415 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310417 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310420 5627 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310427 5627 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310430 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310432 5627 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310435 5627 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310437 5627 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310440 5627 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310442 5627 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:19:39.322120 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310445 5627 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310448 5627 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310450 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310452 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310455 5627 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310457 5627 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310461 5627 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310464 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310466 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310469 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310471 5627 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310475 5627 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310478 5627 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310480 5627 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310483 5627 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310485 5627 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310487 5627 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310490 5627 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310493 5627 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:19:39.322638 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310495 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310498 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310500 5627 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310503 5627 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310505 5627 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310508 5627 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310510 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310513 5627 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310516 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310519 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310521 5627 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310524 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310526 5627 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310529 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310531 5627 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310534 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310537 5627 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310539 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310542 5627 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:19:39.323197 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310546 5627 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310548 5627 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310551 5627 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310554 5627 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310556 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310563 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310567 5627 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310571 5627 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310573 5627 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310576 5627 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310579 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310581 5627 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310584 5627 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310586 5627 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310589 5627 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310591 5627 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310594 5627 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310596 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310599 5627 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:19:39.323655 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310601 5627 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310604 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310607 5627 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310610 5627 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310615 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310619 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310622 5627 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310624 5627 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310627 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310629 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310631 5627 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310634 5627 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310636 5627 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310642 5627 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310644 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310647 5627 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310650 5627 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310653 5627 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310656 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310659 5627 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:19:39.324250 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310661 5627 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310664 5627 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310667 5627 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310669 5627 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310672 5627 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310675 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310677 5627 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310680 5627 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310682 5627 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310685 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.310688 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.315933 5627 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.315956 5627 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.315981 5627 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.316019 5627 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316473 5627 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316489 5627 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316493 5627 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316496 5627 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:19:39.324945 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316499 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316501 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316504 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316508 5627 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316510 5627 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316513 5627 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316516 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316518 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316521 5627 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316524 5627 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316527 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316530 5627 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316533 5627 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316535 5627 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316538 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316541 5627 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316544 5627 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316546 5627 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316549 5627 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316551 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:19:39.325448 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316554 5627 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316556 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316559 5627 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316561 5627 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316564 5627 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316567 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316569 5627 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316571 5627 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316575 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316579 5627 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316585 5627 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316588 5627 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316591 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316594 5627 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316597 5627 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316600 5627 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316602 5627 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316605 5627 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316607 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316610 5627 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:19:39.325955 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316613 5627 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316615 5627 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316618 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316621 5627 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316623 5627 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316626 5627 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316629 5627 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316632 5627 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316634 5627 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316637 5627 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316639 5627 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316642 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316644 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316646 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316649 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316652 5627 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316654 5627 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316656 5627 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316659 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316661 5627 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:19:39.326422 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316664 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316666 5627 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316669 5627 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316671 5627 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316674 5627 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316676 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316679 5627 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316681 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316684 5627 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316686 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316689 5627 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316691 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316694 5627 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316696 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316722 5627 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316725 5627 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316728 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316731 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316734 5627 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:19:39.326911 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316736 5627 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316739 5627 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316742 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316744 5627 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316746 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316749 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316751 5627 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316754 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316756 5627 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316759 5627 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316762 5627 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316764 5627 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316767 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316770 5627 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316773 5627 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316777 5627 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316780 5627 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316783 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316785 5627 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:19:39.327362 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316788 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316790 5627 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316793 5627 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316795 5627 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.316826 5627 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316985 5627 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316990 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316993 5627 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316996 5627 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.316999 5627 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317008 5627 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317011 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317014 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317016 5627 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317019 5627 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317021 5627 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:19:39.327829 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317024 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317027 5627 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317029 5627 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317032 5627 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317035 5627 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317037 5627 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317040 5627 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317042 5627 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317045 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317047 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317049 5627 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317052 5627 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317055 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317057 5627 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317059 5627 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317062 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317064 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317067 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317069 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:19:39.328209 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317071 5627 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317074 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317077 5627 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317080 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317083 5627 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317085 5627 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317088 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317091 5627 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317102 5627 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317106 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317110 5627 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317113 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317116 5627 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317118 5627 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317121 5627 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317124 5627 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317127 5627 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317129 5627 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317132 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:19:39.328650 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317134 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317137 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317139 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317142 5627 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317144 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317147 5627 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317151 5627 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317154 5627 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317156 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317159 5627 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317162 5627 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317165 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317167 5627 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317170 5627 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317172 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317175 5627 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317177 5627 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317180 5627 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317183 5627 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:19:39.329117 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317185 5627 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317188 5627 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317191 5627 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317194 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317197 5627 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317200 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317202 5627 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317204 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317207 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317210 5627 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317212 5627 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317215 5627 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317218 5627 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317220 5627 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317223 5627 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317226 5627 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317228 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317231 5627 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317233 5627 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317237 5627 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:19:39.329581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317240 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317243 5627 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317245 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317248 5627 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317251 5627 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317253 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317256 5627 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317258 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317260 5627 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317263 5627 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317265 5627 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317268 5627 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317271 5627 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317274 5627 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317277 5627 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317280 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317283 5627 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:19:39.330220 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317287 5627 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.317311 5627 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317343 5627 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317348 5627 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317350 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317353 5627 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317356 5627 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317358 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317361 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317363 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317366 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317368 5627 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317371 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317374 5627 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317376 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317381 5627 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 19:19:39.330617 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317386 5627 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317389 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317391 5627 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317395 5627 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317398 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317402 5627 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317404 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317407 5627 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317410 5627 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317412 5627 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317415 5627 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317417 5627 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317420 5627 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317422 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317425 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317427 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317430 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317433 5627 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317436 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 19:19:39.331055 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317438 5627 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317441 5627 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317443 5627 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317446 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317448 5627 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317451 5627 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317453 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317456 5627 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317458 5627 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317461 5627 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317463 5627 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317466 5627 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317470 5627 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317474 5627 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317477 5627 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317479 5627 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317482 5627 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317484 5627 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317487 5627 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 19:19:39.331504 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317489 5627 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317492 5627 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317494 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317497 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317499 5627 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317502 5627 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317504 5627 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317507 5627 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317509 5627 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317511 5627 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317514 5627 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317517 5627 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317519 5627 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317522 5627 feature_gate.go:386] unrecognized feature gate: Example Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317537 5627 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317541 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317543 5627 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317546 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317548 5627 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317551 5627 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 19:19:39.332026 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317553 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317556 5627 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317559 5627 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317561 5627 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317563 5627 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317567 5627 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317571 5627 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317574 5627 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317578 5627 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317582 5627 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317584 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317587 5627 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317591 5627 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317593 5627 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317596 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317598 5627 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317601 5627 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317604 5627 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317606 5627 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 19:19:39.332554 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317609 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317611 5627 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317614 5627 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317616 5627 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317619 5627 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317623 5627 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317626 5627 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317628 5627 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317631 5627 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317633 5627 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317636 5627 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317638 5627 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317641 5627 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317643 5627 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:19:39.317646 5627 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.317726 5627 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.318726 5627 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.318782 5627 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.318971 5627 server.go:1014] "Starting client certificate rotation" Oct 30 19:19:39.333256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.319116 5627 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 19:19:39.334030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.319177 5627 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:13:29 +0000 UTC" deadline="2028-07-12 07:02:01.487389544 +0000 UTC" Oct 30 19:19:39.334030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.319230 5627 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14891h42m22.168162868s" Oct 30 19:19:39.334030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.319438 5627 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:19:39.334030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.320609 5627 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 19:19:39.334030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.322216 5627 log.go:25] "Validated CRI v1 runtime API" Oct 30 19:19:39.334030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.323931 5627 log.go:25] "Validated CRI v1 image API" Oct 30 19:19:39.334030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.324627 5627 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 19:19:39.334030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.326528 5627 fs.go:135] Filesystem UUIDs: map[3133ff10-e4fd-4d43-8770-a9b16ab6886a:/dev/nvme0n1p3 36e46223-66cd-4feb-8c0c-c99c05e7b95d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Oct 30 19:19:39.334294 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.326550 5627 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/082bf2a5b328f2f851966faa4fdc2865a5800fa728bc40e1d55b54cbbe585790/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/082bf2a5b328f2f851966faa4fdc2865a5800fa728bc40e1d55b54cbbe585790/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/09d2070eee1ace4e48896c774b152edd9c3b06af8ec157155f8d2819f80ff2be/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/09d2070eee1ace4e48896c774b152edd9c3b06af8ec157155f8d2819f80ff2be/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/19576ebd5b64807ef06b3b06de660200f5c0a6b6c03872353f82a6fc4bc5abcd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/19576ebd5b64807ef06b3b06de660200f5c0a6b6c03872353f82a6fc4bc5abcd/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/19e0fc085bb97d28b7f06c6722ef3a37f3d8a9efc315cf798759c80ef2fc9709/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/19e0fc085bb97d28b7f06c6722ef3a37f3d8a9efc315cf798759c80ef2fc9709/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/41bc887262dc5cdcb76c55ceae2cdf57d475c065a36aa4b1a413c425e0bbe4b4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/41bc887262dc5cdcb76c55ceae2cdf57d475c065a36aa4b1a413c425e0bbe4b4/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/450f86e9d510006611076f12c8ddb5613d56d965230923dcd308b88084990b17/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/450f86e9d510006611076f12c8ddb5613d56d965230923dcd308b88084990b17/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/55853780a57810149cdbd60dca7709382a1b6235d6a62639cecbb936f95aff1f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/55853780a57810149cdbd60dca7709382a1b6235d6a62639cecbb936f95aff1f/userdata/shm major:0 minor:329 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/63021d1d189dc8b30d057e6df490079c78fb4aefac4c6ec31d5c47d3a3577f1d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/63021d1d189dc8b30d057e6df490079c78fb4aefac4c6ec31d5c47d3a3577f1d/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7cb70437aa5333b50c7c30775d8ac6fffaefb70de87e55420f4848a0452f2027/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7cb70437aa5333b50c7c30775d8ac6fffaefb70de87e55420f4848a0452f2027/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/851adf46f4206e25d21c4eb5044a85e66cb1eb6fd6ed79875e712c311289df9a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/851adf46f4206e25d21c4eb5044a85e66cb1eb6fd6ed79875e712c311289df9a/userdata/shm major:0 minor:58 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8603b15aa833f83b3e0ff835b3cd48635b80f2dbcfe9155e64f06ca04c6ce23f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8603b15aa833f83b3e0ff835b3cd48635b80f2dbcfe9155e64f06ca04c6ce23f/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/898246d70a75b8011907d3e92fc4c618c3cdb94f1a259505bb427c8522697b97/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/898246d70a75b8011907d3e92fc4c618c3cdb94f1a259505bb427c8522697b97/userdata/shm major:0 minor:260 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8f1420f45ced8f5ba994a8774aad9eefbfc3b164d7d8e1a93f03787d3b430730/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8f1420f45ced8f5ba994a8774aad9eefbfc3b164d7d8e1a93f03787d3b430730/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/91e495142ac9377e15c7698fafe3fce5be5b40ac89717b2583489c22d6ab70b9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/91e495142ac9377e15c7698fafe3fce5be5b40ac89717b2583489c22d6ab70b9/userdata/shm major:0 minor:327 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9269626f9c5a62338464b96099da75433559d770a7a2244d87c7ca4d65938eba/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9269626f9c5a62338464b96099da75433559d770a7a2244d87c7ca4d65938eba/userdata/shm major:0 minor:330 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a7c1c487ad6573c94fff690a1770f0fd70a8193dc7f940347646eb30a49360f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a7c1c487ad6573c94fff690a1770f0fd70a8193dc7f940347646eb30a49360f6/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a93f7693bcd7e5c9ee5a7d273f7618ea731544a6d881a66d04290362e78c90a2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a93f7693bcd7e5c9ee5a7d273f7618ea731544a6d881a66d04290362e78c90a2/userdata/shm major:0 minor:267 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aa6056575f3baa76bffb23bb3f3e376a3186d92cb0f5d6814f8adaf61324c0f4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aa6056575f3baa76bffb23bb3f3e376a3186d92cb0f5d6814f8adaf61324c0f4/userdata/shm major:0 minor:314 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/adb804760b0bd20f9bab14ecb77f95d42c392048b3db06630ffb8f95222f9d68/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/adb804760b0bd20f9bab14ecb77f95d42c392048b3db06630ffb8f95222f9d68/userdata/shm major:0 minor:297 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d2b19c163a18477bde18b361096e6b064aff151d42bdfa72301588ec6bcd0b4d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d2b19c163a18477bde18b361096e6b064aff151d42bdfa72301588ec6bcd0b4d/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dec3ac18faa447e3b545f7a27d0fb27fc7d7eae0d0bf55b8e7a25173514f3197/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dec3ac18faa447e3b545f7a27d0fb27fc7d7eae0d0bf55b8e7a25173514f3197/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~projected/kube-api-access-f7wtf:{mountpoint:/var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~projected/kube-api-access-f7wtf major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:253 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~projected/kube-api-access-mw4qf:{mountpoint:/var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~projected/kube-api-access-mw4qf major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f59ba06-3da0-4638-9cd1-df7a19645b1d/volumes/kubernetes.io~projected/kube-api-access-q796l:{mountpoint:/var/lib/kubelet/pods/2f59ba06-3da0-4638-9cd1-df7a19645b1d/volumes/kubernetes.io~projected/kube-api-access-q796l major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f59ba06-3da0-4638-9cd1-df7a19645b1d/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/2f59ba06-3da0-4638-9cd1-df7a19645b1d/volumes/kubernetes.io~secret/metrics-certs major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~projected/kube-api-access-spgqv:{mountpoint:/var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~projected/kube-api-access-spgqv major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volumes/kubernetes.io~projected/kube-api-access-7rrcs:{mountpoint:/var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volumes/kubernetes.io~projected/kube-api-access-7rrcs major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52f645a5-26c7-4cad-9834-71eacd7c9a88/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/52f645a5-26c7-4cad-9834-71eacd7c9a88/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6afdc8a2-0be6-424f-ba13-99a3fcf436dc/volumes/kubernetes.io~projected/kube-api-access-7bglv:{mountpoint:/var/lib/kubelet/pods/6afdc8a2-0be6-424f-ba13-99a3fcf436dc/volumes/kubernetes.io~projected/kube-api-access-7bglv major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6afdc8a2-0be6-424f-ba13-99a3fcf436dc/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/6afdc8a2-0be6-424f-ba13-99a3fcf436dc/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/756b72aa-da1a-4e51-8adf-df95022277d8/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/756b72aa-da1a-4e51-8adf-df95022277d8/volumes/kubernetes.io~secret/agent-certs major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a008f757-b4b8-48a7-bb30-ffa1c1d39b75/volumes/kubernetes.io~projected/kube-api-access-k9994:{mountpoint:/var/lib/kubelet/pods/a008f757-b4b8-48a7-bb30-ffa1c1d39b75/volumes/kubernetes.io~projected/kube-api-access-k9994 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a489b325-efa4-4d23-b7ae-978c813aa93e/volumes/kubernetes.io~projected/kube-api-access-8k59h:{mountpoint:/var/lib/kubelet/pods/a489b325-efa4-4d23-b7ae-978c813aa93e/volumes/kubernetes.io~projected/kube-api-access-8k59h major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a5fac3e1-506c-4ce0-b992-9108addc1e81/volumes/kubernetes.io~projected/kube-api-access-5h77j:{mountpoint:/var/lib/kubelet/pods/a5fac3e1-506c-4ce0-b992-9108addc1e81/volumes/kubernetes.io~projected/kube-api-access-5h77j major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a5fac3e1-506c-4ce0-b992-9108addc1e81/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/a5fac3e1-506c-4ce0-b992-9108addc1e81/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac144cc0-bc7d-4430-a53b-194bcfb77145/volumes/kubernetes.io~projected/kube-api-access-b2hwl:{mountpoint:/var/lib/kubelet/pods/ac144cc0-bc7d-4430-a53b-194bcfb77145/volumes/kubernetes.io~projected/kube-api-access-b2hwl major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/acf1ec3f-be10-4166-ba73-8994e5d8d7a1/volumes/kubernetes.io~projected/kube-api-access-mp899:{mountpoint:/var/lib/kubelet/pods/acf1ec3f-be10-4166-ba73-8994e5d8d7a1/volumes/kubernetes.io~projected/kube-api-access-mp899 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/acf1ec3f-be10-4166-ba73-8994e5d8d7a1/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/acf1ec3f-be10-4166-ba73-8994e5d8d7a1/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~projected/kube-api-access-n6hdf:{mountpoint:/var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~projected/kube-api-access-n6hdf major:0 minor:266 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c956bc7a-44f6-4d27-b9be-0d73f6dedd01/volumes/kubernetes.io~projected/kube-api-access-w26kz:{mountpoint:/var/lib/kubelet/pods/c956bc7a-44f6-4d27-b9be-0d73f6dedd01/volumes/kubernetes.io~projected/kube-api-access-w26kz major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d20919c9-e2a8-4e07-89ea-bc7cc726444c/volumes/kubernetes.io~projected/kube-api-access-zz75l:{mountpoint:/var/lib/kubelet/pods/d20919c9-e2a8-4e07-89ea-bc7cc726444c/volumes/kubernetes.io~projected/kube-api-access-zz75l major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcecdd80-d19c-4b8d-b38f-7067c98c7f98/volumes/kubernetes.io~projected/kube-api-access-qj9kx:{mountpoint:/var/lib/kubelet/pods/dcecdd80-d19c-4b8d-b38f-7067c98c7f98/volumes/kubernetes.io~projected/kube-api-access-qj9kx major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~projected/kube-api-access-2ssdd:{mountpoint:/var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~projected/kube-api-access-2ssdd major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:308 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb0fad83-891c-4219-a590-846d4ca0089e/volumes/kubernetes.io~projected/kube-api-access-x74g6:{mountpoint:/var/lib/kubelet/pods/eb0fad83-891c-4219-a590-846d4ca0089e/volumes/kubernetes.io~projected/kube-api-access-x74g6 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~projected/kube-api-access-2gtcg:{mountpoint:/var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~projected/kube-api-access-2gtcg major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:240 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/7349ddce281b1b4832ea84a5175121786cd9f1d1ae0630cf88b7c94560d23b55/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/b000eaf694c9f5d4e2cf30d3bf8f4be381edacd2dd3ee855b29a682cc9b5df60/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/6f8f03b6f5ba2b4817a1521232f03652c4eb74cbf7d54e3a4e70001e7a677091/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/5562f4f6594e40e9945599ca4757024b6141ff6938cf1d294352ce90edeb40b4/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/d9d2ac971fd969486e45cb6585ea1d808f7ed795e637e10336c5a14b67ad5daf/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/6a9bccb0f607425fd50e4dcd6189561952dcda9c5ca39efef8a95b1327c2de91/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/fa6c46eceb04f8b1eb422eb2db89d93239ad0809d0a71bdc301e62394d8989f1/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/e45330d9dbcd32144b5e8c379b511df3442867584774c5c6eb0c34b19ebe6d14/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/a12bea04d5adca537cfc8f59893f06b4b794c3584763205ea7623983a0f1745a/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/8c39c8fda4a579e56353e7808b9c5f4c18617c97550ab3470c0cc498d0b103af/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/22f20b88ec03a54954e540c81df57165692c56c56efd40932228e298b894bd31/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/d41a92c870db2e117ece4c48977aa8705ac15c2057ec86ade322523cc21806f5/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-151:{mountpoint:/var/lib/containers/storage/overlay/c82cce9343facde8ad58cc41903122d8163378da7ebe3e10a5e681936586158a/merged major:0 minor:151 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/1c193d7a2a2f4d45362aaeb6300dbd70b5306ecbdc1a21ed75435c5302258f2e/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/cb4956b7f87e74c12d7c29ba84c1cc2a331e3fae4e1e9bd3855d1cbc6653935b/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/276c7136ec88636eed70b6ca2c44d1b08b74698f2c16eb5dee05d38fe6020073/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/a78aefb483abc39f2d93ba3f19adf904c5b28d3b80af610ff9fdcdb73b061764/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/cd406db27c264a1aa92c9353cb09ab155f697413947367f432a50c641160d80a/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/be588c4f9df741b7eaf0cc0915f5c5d1180436c0551ca59cb28b9df9325cb1d9/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/8c66bb89d1945cac1e8cd1368ec02acfc26985fd0528976cdb6f0ab8616ca99d/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/abca116bec451c65f146ca40eb197f770440aed9a71f8e569872c3c1f52c5724/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/02653ebe8fa4edbaeb6adb14c5e305e4e0c4f47515df3ed5ec92b17d1cdd11ca/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/670ef561a4b32613f707ad46db7c7f911bc51aa882d357f4abbffc5cf1eb8140/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/de8e4b329e8e0928ace841113f851b619ea768d39a165aa3a9cf57dadc936e2b/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/db9dcf9429ddd017ab3bf9fdc97a2b176997feb73fff00dbe13f3f9d81af64d8/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/b717060e843a5d80803eb2c8aa841bdb7d719549c6c1487cb221481d0eb14440/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/c5c7215058f3b90973e5bf90a227b0e92541a55bd3f9afdf9d2f526a24070cdc/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/3647e2607a4af8908c49fc6037ddbd73edd44714c3af3c49d1be041641a95ae2/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-244:{mountpoint:/var/lib/containers/storage/overlay/05ed8f6c2ca651bbc86a9460d3f0dd38e3dc184774ed473be8b05ef8d6c3a6fe/merged major:0 minor:244 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/76e2e09e8b779c590fd6a3e7fe8d6f492fcbf8d7ef00dc8643a198f87a768015/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/8c9b7f9d04197d81b7965ad15e67886ec8b1d970da2b283e62aab13e2caa3d5c/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/b4e326507c50144529bee4f3f8793a5c6b58a735ccaf3ccb0d2c1ec9c277bfd0/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/fdea0ba8c691d5ee5b15889af7b65d81ac1f50e78c92a21af4aec7adc213932c/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/77214b6bc4881045a746880f97b1f8b169a50d9d6e1413f063ec70a4ee48cebe/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/c18809f42c72c16ee1c95791d711a17c1e8b9e3bdff9b7c283a7099609e0f501/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/2c78fa6ea2695bbfe6c692010546d277f530c8888620ac1ec929fcf4d82e75c6/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/ef203f874d6e5f516ca6d9fcdd4ab766a9d873c2028834bd8d0cef8dbced9bd5/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/dd4c2a185e99f4a5c7f4a04d868f288ebfd75ff6dd53a64b566fe6e1d80e7e2b/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/83b544f63f9c7950184747fecfddadb8ea43ffd1a282b9449932eeadbcda9b1b/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/815cb24779ba5d7a517c3cea2aff1e62b4bcf83507c18431a5a23002851409c2/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/5cc272d1b58c4b4b6dd7cfea2472d470459282b69c24d74e19d17e8e7a4e5818/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/8dc14a0b30a1b7455ccebe9b35db28a2387af46542950d002dac89a9a1f50da0/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-333:{mountpoint:/var/lib/containers/storage/overlay/b72191df042c60c6ead3e01f808827abe63e64273269a4a499027b382fdbc1db/merged major:0 minor:333 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/d601f8c04eb806403d84ed35fd14af607f239d43870e7e3e7a5780baba39ce02/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-337:{mountpoint:/var/lib/containers/storage/overlay/e308a8181756556fd9976c8f728eb7dd5b5b92b902a6937531cc0b7a0dfeb049/merged major:0 minor:337 fsType:overlay blockSize:0} overlay_0-339:{mountpoint:/var/lib/containers/storage/overlay/c26a36032d3f183fa2d41850a838870c80d27362a957773f144372c30f848dca/merged major:0 minor:339 fsType:overlay blockSize:0} overlay_0-341:{mountpoint:/var/lib/containers/storage/overlay/f7a24827cffd1b79ee719d5f54b2a780fa4ed966d0642b0861d1b118e3004b66/merged major:0 minor:341 fsType:overlay blockSize:0} overlay_0-347:{mountpoint:/var/lib/containers/storage/overlay/9d63a66d23253076b9509ccbb5554f9f91ee53b732ea4af881a197a08467589d/merged major:0 minor:347 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/198ffbba8da3187c2e7663c4b7c7e71b33974832d01d97d01666d775955e0592/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-60:{mountpoint:/var/lib/containers/storage/overlay/87aefadaf08db7f7d4820d8a5e4e7723f5115c7ec3629cdb207fd58156ee9539/merged major:0 minor:60 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/2b1a460f00a1944370067228fcf8cd35573a3cc94144a6072f3dc0563e0e15cc/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/25f1d5b4f069d3268bc68f95d2d7c1f4d4e2ed68ac96a8f6c977c245b323e585/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/b8b1c99baf15b76a35697c7bfc40c39685fe9beaf5695fe885242ff5588a0766/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/02da5b4beeb6d7b038b10e1b7422bcab717f6bc337d80749c5aeb347ea7595f1/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/9ab029f892f2a2dec33e6ea2979d418c716859ffd8951b97b7498b91562bc329/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/1730be2638d0e5cccf33ab7feab24bf9eac839fe8c3ca8334747057ea336e3e1/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/7b870f7058f04b71821fadc2376eeb0a00e821b7eeb079f5bb5799b0c0b53e02/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/34a9adea6f49cd6b40dbafd5d2f549642cf4799ca608e6f11bf79ac035abc8c7/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/3279ddbca1db19067f06013228ccfb84e9cb8279069424ee38a860efecaa7789/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 19:19:39.335795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.335382 5627 manager.go:217] Machine: {Timestamp:2026-10-30 19:19:39.334817897 +0000 UTC m=+0.061635186 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3118315 MemoryCapacity:32812003328 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27e24058190940ad438187542a2b68 SystemUUID:ec27e240-5819-0940-ad43-8187542a2b68 BootID:0def5452-661a-443c-ae13-8ba37d045f4a Filesystems:[{Device:/var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:50 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/dcecdd80-d19c-4b8d-b38f-7067c98c7f98/volumes/kubernetes.io~projected/kube-api-access-qj9kx DeviceMajor:0 DeviceMinor:66 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/41bc887262dc5cdcb76c55ceae2cdf57d475c065a36aa4b1a413c425e0bbe4b4/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/52f645a5-26c7-4cad-9834-71eacd7c9a88/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:325 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/2f59ba06-3da0-4638-9cd1-df7a19645b1d/volumes/kubernetes.io~projected/kube-api-access-q796l DeviceMajor:0 DeviceMinor:55 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~projected/kube-api-access-n6hdf DeviceMajor:0 DeviceMinor:266 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-60 DeviceMajor:0 DeviceMinor:60 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8f1420f45ced8f5ba994a8774aad9eefbfc3b164d7d8e1a93f03787d3b430730/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6afdc8a2-0be6-424f-ba13-99a3fcf436dc/volumes/kubernetes.io~projected/kube-api-access-7bglv DeviceMajor:0 DeviceMinor:202 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:265 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac144cc0-bc7d-4430-a53b-194bcfb77145/volumes/kubernetes.io~projected/kube-api-access-b2hwl DeviceMajor:0 DeviceMinor:53 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6afdc8a2-0be6-424f-ba13-99a3fcf436dc/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:253 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~projected/kube-api-access-f7wtf DeviceMajor:0 DeviceMinor:259 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:290 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~projected/kube-api-access-2gtcg DeviceMajor:0 DeviceMinor:241 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dec3ac18faa447e3b545f7a27d0fb27fc7d7eae0d0bf55b8e7a25173514f3197/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-151 DeviceMajor:0 DeviceMinor:151 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a93f7693bcd7e5c9ee5a7d273f7618ea731544a6d881a66d04290362e78c90a2/userdata/shm DeviceMajor:0 DeviceMinor:267 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-337 DeviceMajor:0 DeviceMinor:337 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/14ef0160-1e62-4a92-8158-bdd45eebaeab/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:258 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~projected/kube-api-access-2ssdd DeviceMajor:0 DeviceMinor:313 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8603b15aa833f83b3e0ff835b3cd48635b80f2dbcfe9155e64f06ca04c6ce23f/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a7c1c487ad6573c94fff690a1770f0fd70a8193dc7f940347646eb30a49360f6/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aa6056575f3baa76bffb23bb3f3e376a3186d92cb0f5d6814f8adaf61324c0f4/userdata/shm DeviceMajor:0 DeviceMinor:314 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-341 DeviceMajor:0 DeviceMinor:341 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~projected/kube-api-access-spgqv DeviceMajor:0 DeviceMinor:296 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:312 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/2f59ba06-3da0-4638-9cd1-df7a19645b1d/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:320 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~projected/kube-api-access-mw4qf DeviceMajor:0 DeviceMinor:54 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/63021d1d189dc8b30d057e6df490079c78fb4aefac4c6ec31d5c47d3a3577f1d/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/19576ebd5b64807ef06b3b06de660200f5c0a6b6c03872353f82a6fc4bc5abcd/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d20919c9-e2a8-4e07-89ea-bc7cc726444c/volumes/kubernetes.io~projected/kube-api-access-zz75l DeviceMajor:0 DeviceMinor:326 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9269626f9c5a62338464b96099da75433559d770a7a2244d87c7ca4d65938eba/userdata/shm DeviceMajor:0 DeviceMinor:330 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-333 DeviceMajor:0 DeviceMinor:333 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/082bf2a5b328f2f851966faa4fdc2865a5800fa728bc40e1d55b54cbbe585790/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb0fad83-891c-4219-a590-846d4ca0089e/volumes/kubernetes.io~projected/kube-api-access-x74g6 DeviceMajor:0 DeviceMinor:67 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/a489b325-efa4-4d23-b7ae-978c813aa93e/volumes/kubernetes.io~projected/kube-api-access-8k59h DeviceMajor:0 DeviceMinor:57 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/acf1ec3f-be10-4166-ba73-8994e5d8d7a1/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:236 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/adb804760b0bd20f9bab14ecb77f95d42c392048b3db06630ffb8f95222f9d68/userdata/shm DeviceMajor:0 DeviceMinor:297 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/55853780a57810149cdbd60dca7709382a1b6235d6a62639cecbb936f95aff1f/userdata/shm DeviceMajor:0 DeviceMinor:329 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-347 DeviceMajor:0 DeviceMinor:347 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3f35fd1e-89aa-430a-8198-703bc598c411/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:295 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:308 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/91e495142ac9377e15c7698fafe3fce5be5b40ac89717b2583489c22d6ab70b9/userdata/shm DeviceMajor:0 DeviceMinor:327 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/851adf46f4206e25d21c4eb5044a85e66cb1eb6fd6ed79875e712c311289df9a/userdata/shm DeviceMajor:0 DeviceMinor:58 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a5fac3e1-506c-4ce0-b992-9108addc1e81/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/898246d70a75b8011907d3e92fc4c618c3cdb94f1a259505bb427c8522697b97/userdata/shm DeviceMajor:0 DeviceMinor:260 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-339 DeviceMajor:0 DeviceMinor:339 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7cb70437aa5333b50c7c30775d8ac6fffaefb70de87e55420f4848a0452f2027/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/19e0fc085bb97d28b7f06c6722ef3a37f3d8a9efc315cf798759c80ef2fc9709/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d2b19c163a18477bde18b361096e6b064aff151d42bdfa72301588ec6bcd0b4d/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:240 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/a008f757-b4b8-48a7-bb30-ffa1c1d39b75/volumes/kubernetes.io~projected/kube-api-access-k9994 DeviceMajor:0 DeviceMinor:56 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/volumes/kubernetes.io~projected/kube-api-access-7rrcs DeviceMajor:0 DeviceMinor:52 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/09d2070eee1ace4e48896c774b152edd9c3b06af8ec157155f8d2819f80ff2be/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/a5fac3e1-506c-4ce0-b992-9108addc1e81/volumes/kubernetes.io~projected/kube-api-access-5h77j DeviceMajor:0 DeviceMinor:203 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16405999616 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/acf1ec3f-be10-4166-ba73-8994e5d8d7a1/volumes/kubernetes.io~projected/kube-api-access-mp899 DeviceMajor:0 DeviceMinor:204 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/450f86e9d510006611076f12c8ddb5613d56d965230923dcd308b88084990b17/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:264 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/c956bc7a-44f6-4d27-b9be-0d73f6dedd01/volumes/kubernetes.io~projected/kube-api-access-w26kz DeviceMajor:0 DeviceMinor:51 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-244 DeviceMajor:0 DeviceMinor:244 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ad2d631-46c4-42d5-8c8d-a0db93319a5e/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/756b72aa-da1a-4e51-8adf-df95022277d8/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:49 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:19576ebd5b64807 MacAddress:be:13:ba:6d:b4:84 Speed:10000 Mtu:8901} {Name:450f86e9d510006 MacAddress:be:61:07:c4:68:7c Speed:10000 Mtu:8901} {Name:55853780a578101 MacAddress:da:84:d7:c4:79:a3 Speed:10000 Mtu:8901} {Name:898246d70a75b80 MacAddress:2e:c7:32:2d:37:13 Speed:10000 Mtu:8901} {Name:91e495142ac9377 MacAddress:ee:72:c8:56:4e:ec Speed:10000 Mtu:8901} {Name:9269626f9c5a623 MacAddress:36:73:64:41:56:81 Speed:10000 Mtu:8901} {Name:a7c1c487ad6573c MacAddress:b6:a4:cb:4f:9c:88 Speed:10000 Mtu:8901} {Name:a93f7693bcd7e5c MacAddress:de:ac:04:d8:66:5c Speed:10000 Mtu:8901} {Name:aa6056575f3baa7 MacAddress:56:ab:04:93:bd:b8 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:ff:20:f1:1d Speed:0 Mtu:9001} {Name:br-int MacAddress:56:96:5a:9b:1c:21 Speed:0 Mtu:8901} {Name:d2b19c163a18477 MacAddress:4e:70:54:c8:ad:d1 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:ff:20:f1:1d Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:ae:04:c2:c1:0d:a1 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:2e:3e:85:e7:31:b2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812003328 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 19:19:39.335795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.335798 5627 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 19:19:39.336119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.335852 5627 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 19:19:39.336243 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336216 5627 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 19:19:39.336400 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336244 5627 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-211.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 19:19:39.336475 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336404 5627 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 19:19:39.336475 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336417 5627 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 19:19:39.336475 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336431 5627 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 19:19:39.336475 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336459 5627 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 19:19:39.336635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336600 5627 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 19:19:39.336834 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336820 5627 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 19:19:39.336965 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.336954 5627 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 19:19:39.337021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.337002 5627 kubelet.go:493] "Attempting to sync node with API server" Oct 30 19:19:39.337068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.337023 5627 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 19:19:39.337068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.337037 5627 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 19:19:39.337068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.337051 5627 kubelet.go:405] "Adding apiserver pod source" Oct 30 19:19:39.337068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.337064 5627 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 19:19:39.337759 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.337742 5627 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 19:19:39.337759 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.337757 5627 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 19:19:39.338397 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.338372 5627 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 19:19:39.341844 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.341829 5627 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 19:19:39.341844 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.341842 5627 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 19:19:39.341967 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.341957 5627 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 19:19:39.342168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342157 5627 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 19:19:39.342199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342187 5627 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 19:19:39.342319 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342308 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 19:19:39.342349 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342331 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 19:19:39.342349 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342340 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 19:19:39.342404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342350 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 19:19:39.342404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342361 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 19:19:39.342404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342368 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 19:19:39.342404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342375 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 19:19:39.342404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342382 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 19:19:39.342404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342390 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 19:19:39.342404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342397 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 19:19:39.342642 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342409 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 19:19:39.342642 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342425 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 19:19:39.342642 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342452 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 19:19:39.342642 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.342459 5627 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 19:19:39.346732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.346718 5627 server.go:1263] "Started kubelet" Oct 30 19:19:39.346872 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.346837 5627 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 19:19:39.347656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.347086 5627 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 19:19:39.347656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.347535 5627 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 19:19:39.347482 ip-10-0-139-211 systemd[1]: Started Kubernetes Kubelet. Oct 30 19:19:39.348920 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.348826 5627 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 19:19:39.350401 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.349984 5627 server.go:317] "Adding debug handlers to kubelet server" Oct 30 19:19:39.356836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.356791 5627 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:19:39.359934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.359484 5627 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 19:19:39.359934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.359531 5627 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 19:19:39.359934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.359589 5627 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 19:19:39.359934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.359604 5627 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 19:19:39.359934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.359599 5627 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:13:30 +0000 UTC" deadline="2028-07-19 08:11:27.319906896 +0000 UTC" Oct 30 19:19:39.359934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.359619 5627 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15060h51m47.960290955s" Oct 30 19:19:39.359934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.359659 5627 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 19:19:39.360317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.360104 5627 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:19:39.361995 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.361978 5627 factory.go:153] Registering CRI-O factory Oct 30 19:19:39.362080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.361999 5627 factory.go:223] Registration of the crio container factory successfully Oct 30 19:19:39.362080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362060 5627 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 19:19:39.362080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362069 5627 factory.go:55] Registering systemd factory Oct 30 19:19:39.362080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362074 5627 factory.go:223] Registration of the systemd container factory successfully Oct 30 19:19:39.362214 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362090 5627 factory.go:103] Registering Raw factory Oct 30 19:19:39.362214 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362103 5627 manager.go:1199] Started watching for new ooms in manager Oct 30 19:19:39.362214 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362142 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e646b5f0-ae93-4f27-a32f-78bbd45b837e" volumeName="kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert" seLinuxMountContext="" Oct 30 19:19:39.362304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362210 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb0fad83-891c-4219-a590-846d4ca0089e" volumeName="kubernetes.io/empty-dir/eb0fad83-891c-4219-a590-846d4ca0089e-tmp-dir" seLinuxMountContext="" Oct 30 19:19:39.362304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362232 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ad2d631-46c4-42d5-8c8d-a0db93319a5e" volumeName="kubernetes.io/projected/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-kube-api-access-mw4qf" seLinuxMountContext="" Oct 30 19:19:39.362304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362247 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f59ba06-3da0-4638-9cd1-df7a19645b1d" volumeName="kubernetes.io/projected/2f59ba06-3da0-4638-9cd1-df7a19645b1d-kube-api-access-q796l" seLinuxMountContext="" Oct 30 19:19:39.362304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362260 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6afdc8a2-0be6-424f-ba13-99a3fcf436dc" volumeName="kubernetes.io/projected/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-api-access-7bglv" seLinuxMountContext="" Oct 30 19:19:39.362304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362277 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a489b325-efa4-4d23-b7ae-978c813aa93e" volumeName="kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 19:19:39.362304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362290 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4da4805-c7d7-408c-9f75-cd124fb938e9" volumeName="kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362305 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e646b5f0-ae93-4f27-a32f-78bbd45b837e" volumeName="kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362323 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e646b5f0-ae93-4f27-a32f-78bbd45b837e" volumeName="kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362339 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e646b5f0-ae93-4f27-a32f-78bbd45b837e" volumeName="kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362354 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a008f757-b4b8-48a7-bb30-ffa1c1d39b75" volumeName="kubernetes.io/configmap/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-serviceca" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362368 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ad2d631-46c4-42d5-8c8d-a0db93319a5e" volumeName="kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-tmp" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362385 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a3d3354-c4de-4bf4-8d99-80808fe2e6f9" volumeName="kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-script-lib" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362394 5627 manager.go:319] Starting recovery of all containers Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362400 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a3d3354-c4de-4bf4-8d99-80808fe2e6f9" volumeName="kubernetes.io/projected/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-kube-api-access-7rrcs" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362417 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6afdc8a2-0be6-424f-ba13-99a3fcf436dc" volumeName="kubernetes.io/empty-dir/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-data-volume" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362429 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a008f757-b4b8-48a7-bb30-ffa1c1d39b75" volumeName="kubernetes.io/projected/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-kube-api-access-k9994" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362441 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7e1781-2624-4b67-acbe-a36a05e238a9" volumeName="kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp" seLinuxMountContext="" Oct 30 19:19:39.362481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362470 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14ef0160-1e62-4a92-8158-bdd45eebaeab" volumeName="kubernetes.io/configmap/14ef0160-1e62-4a92-8158-bdd45eebaeab-metrics-client-ca" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362484 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14ef0160-1e62-4a92-8158-bdd45eebaeab" volumeName="kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362500 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="756b72aa-da1a-4e51-8adf-df95022277d8" volumeName="kubernetes.io/secret/756b72aa-da1a-4e51-8adf-df95022277d8-agent-certs" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362512 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6afdc8a2-0be6-424f-ba13-99a3fcf436dc" volumeName="kubernetes.io/secret/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-insights-runtime-extractor-tls" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362524 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="acf1ec3f-be10-4166-ba73-8994e5d8d7a1" volumeName="kubernetes.io/projected/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-kube-api-access-mp899" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362537 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4da4805-c7d7-408c-9f75-cd124fb938e9" volumeName="kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362551 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4da4805-c7d7-408c-9f75-cd124fb938e9" volumeName="kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362565 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d20919c9-e2a8-4e07-89ea-bc7cc726444c" volumeName="kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362577 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb0fad83-891c-4219-a590-846d4ca0089e" volumeName="kubernetes.io/projected/eb0fad83-891c-4219-a590-846d4ca0089e-kube-api-access-x74g6" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362590 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ad2d631-46c4-42d5-8c8d-a0db93319a5e" volumeName="kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-tuned" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362605 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a3d3354-c4de-4bf4-8d99-80808fe2e6f9" volumeName="kubernetes.io/secret/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362633 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4da4805-c7d7-408c-9f75-cd124fb938e9" volumeName="kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362646 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f35fd1e-89aa-430a-8198-703bc598c411" volumeName="kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-tls" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362661 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a489b325-efa4-4d23-b7ae-978c813aa93e" volumeName="kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-binary-copy" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362675 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcecdd80-d19c-4b8d-b38f-7067c98c7f98" volumeName="kubernetes.io/projected/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-kube-api-access-qj9kx" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362690 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac144cc0-bc7d-4430-a53b-194bcfb77145" volumeName="kubernetes.io/projected/ac144cc0-bc7d-4430-a53b-194bcfb77145-kube-api-access-b2hwl" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362720 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c956bc7a-44f6-4d27-b9be-0d73f6dedd01" volumeName="kubernetes.io/projected/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kube-api-access-w26kz" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362735 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7e1781-2624-4b67-acbe-a36a05e238a9" volumeName="kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362748 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7e1781-2624-4b67-acbe-a36a05e238a9" volumeName="kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362765 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="756b72aa-da1a-4e51-8adf-df95022277d8" volumeName="kubernetes.io/configmap/756b72aa-da1a-4e51-8adf-df95022277d8-konnectivity-ca" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362777 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7e1781-2624-4b67-acbe-a36a05e238a9" volumeName="kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362789 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a489b325-efa4-4d23-b7ae-978c813aa93e" volumeName="kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362801 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcecdd80-d19c-4b8d-b38f-7067c98c7f98" volumeName="kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cni-binary-copy" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362816 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e646b5f0-ae93-4f27-a32f-78bbd45b837e" volumeName="kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362828 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f35fd1e-89aa-430a-8198-703bc598c411" volumeName="kubernetes.io/empty-dir/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-textfile" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362844 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f35fd1e-89aa-430a-8198-703bc598c411" volumeName="kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362857 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a3d3354-c4de-4bf4-8d99-80808fe2e6f9" volumeName="kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-env-overrides" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362871 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="acf1ec3f-be10-4166-ba73-8994e5d8d7a1" volumeName="kubernetes.io/configmap/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-config-volume" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362887 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4da4805-c7d7-408c-9f75-cd124fb938e9" volumeName="kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362901 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f35fd1e-89aa-430a-8198-703bc598c411" volumeName="kubernetes.io/projected/3f35fd1e-89aa-430a-8198-703bc598c411-kube-api-access-spgqv" seLinuxMountContext="" Oct 30 19:19:39.362916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362913 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a5fac3e1-506c-4ce0-b992-9108addc1e81" volumeName="kubernetes.io/secret/a5fac3e1-506c-4ce0-b992-9108addc1e81-cert" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362929 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcecdd80-d19c-4b8d-b38f-7067c98c7f98" volumeName="kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-daemon-config" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362943 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e646b5f0-ae93-4f27-a32f-78bbd45b837e" volumeName="kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362957 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7e1781-2624-4b67-acbe-a36a05e238a9" volumeName="kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362971 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4da4805-c7d7-408c-9f75-cd124fb938e9" volumeName="kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362986 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e646b5f0-ae93-4f27-a32f-78bbd45b837e" volumeName="kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.362998 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7e1781-2624-4b67-acbe-a36a05e238a9" volumeName="kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363012 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7e1781-2624-4b67-acbe-a36a05e238a9" volumeName="kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363025 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52f645a5-26c7-4cad-9834-71eacd7c9a88" volumeName="kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363063 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6afdc8a2-0be6-424f-ba13-99a3fcf436dc" volumeName="kubernetes.io/configmap/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-rbac-proxy-cm" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363076 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a5fac3e1-506c-4ce0-b992-9108addc1e81" volumeName="kubernetes.io/projected/a5fac3e1-506c-4ce0-b992-9108addc1e81-kube-api-access-5h77j" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363088 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="acf1ec3f-be10-4166-ba73-8994e5d8d7a1" volumeName="kubernetes.io/empty-dir/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-tmp-dir" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363100 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="acf1ec3f-be10-4166-ba73-8994e5d8d7a1" volumeName="kubernetes.io/secret/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-metrics-tls" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363114 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e646b5f0-ae93-4f27-a32f-78bbd45b837e" volumeName="kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363129 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14ef0160-1e62-4a92-8158-bdd45eebaeab" volumeName="kubernetes.io/projected/14ef0160-1e62-4a92-8158-bdd45eebaeab-kube-api-access-f7wtf" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363142 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14ef0160-1e62-4a92-8158-bdd45eebaeab" volumeName="kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-tls" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363154 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f59ba06-3da0-4638-9cd1-df7a19645b1d" volumeName="kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363167 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f35fd1e-89aa-430a-8198-703bc598c411" volumeName="kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-metrics-client-ca" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363180 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f35fd1e-89aa-430a-8198-703bc598c411" volumeName="kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-accelerators-collector-config" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363194 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a3d3354-c4de-4bf4-8d99-80808fe2e6f9" volumeName="kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-config" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363209 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac144cc0-bc7d-4430-a53b-194bcfb77145" volumeName="kubernetes.io/configmap/ac144cc0-bc7d-4430-a53b-194bcfb77145-iptables-alerter-script" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363221 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4da4805-c7d7-408c-9f75-cd124fb938e9" volumeName="kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363232 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a489b325-efa4-4d23-b7ae-978c813aa93e" volumeName="kubernetes.io/projected/a489b325-efa4-4d23-b7ae-978c813aa93e-kube-api-access-8k59h" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363245 5627 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4da4805-c7d7-408c-9f75-cd124fb938e9" volumeName="kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf" seLinuxMountContext="" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363258 5627 reconstruct.go:97] "Volume reconstruction finished" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363266 5627 reconciler.go:29] "Reconciler: start to sync state" Oct 30 19:19:39.363897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.363850 5627 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:19:39.370789 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.368424 5627 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 19:19:39.382596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.382571 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:19:39.385637 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.385615 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:19:39.389089 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.389056 5627 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 19:19:39.390583 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.390560 5627 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 19:19:39.390774 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.390665 5627 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 19:19:39.390932 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.390919 5627 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 19:19:39.390985 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:19:39.390961 5627 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 19:19:39.395935 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.395911 5627 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 19:19:39.423095 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.423069 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:19:39.423440 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.423411 5627 generic.go:356] "Generic (PLEG): container finished" podID="4a3d3354-c4de-4bf4-8d99-80808fe2e6f9" containerID="5410b5ec2754c6e216d6d5da1985fcf37c1c08f0aea87cf0c6d06b2f8975ed90" exitCode=1 Oct 30 19:19:39.430324 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.430288 5627 generic.go:356] "Generic (PLEG): container finished" podID="3f35fd1e-89aa-430a-8198-703bc598c411" containerID="f139de66cb6a0d9a35812bdd2da2577157a9cc1fc3787ddf32958404b15c2aa8" exitCode=0 Oct 30 19:19:39.437637 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.437606 5627 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="ddefcc5940c74de9e8013cf0ae0139098931de79de46364a774ce1926dcce4bc" exitCode=0 Oct 30 19:19:39.437753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.437639 5627 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="1922bce96665d6593de528b72bcc862bffdc9f3cd612edc7b377c5f6624a0cef" exitCode=0 Oct 30 19:19:39.437753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.437653 5627 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="09e98c52a1d3f4099a890487eaad0680f23bd33a1d8169bd0b1a83cfc73b555e" exitCode=0 Oct 30 19:19:39.437753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.437668 5627 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="341392f3a5a7d9875e75af507402fffafb26970fcc4b483ddf0acdf3e526ee7c" exitCode=0 Oct 30 19:19:39.437753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.437680 5627 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="32147fe6a3dd373d3637067ad3648cb4715fe341cd8aab3244031ce070fa7165" exitCode=0 Oct 30 19:19:39.437753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.437690 5627 generic.go:356] "Generic (PLEG): container finished" podID="a489b325-efa4-4d23-b7ae-978c813aa93e" containerID="df618cd317c940a46775c5871f63f5b8aedb298f1b94f43f9420b08634f062d7" exitCode=0 Oct 30 19:19:39.440834 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.440810 5627 generic.go:356] "Generic (PLEG): container finished" podID="5c2829c8c9b99a6ca60457b5349d0af7" containerID="57540385dbe44a2ba598f090f4ff8930108a909c34ec3b54cd1202f5c5befdbf" exitCode=0 Oct 30 19:19:39.452344 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.452326 5627 manager.go:324] Recovery completed Oct 30 19:19:39.473337 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473284 5627 cpu_manager.go:226] "Starting" policy="none" Oct 30 19:19:39.473337 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473301 5627 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 19:19:39.473337 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473317 5627 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 19:19:39.473486 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473445 5627 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 19:19:39.473486 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473457 5627 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 19:19:39.473486 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473474 5627 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 19:19:39.473486 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473479 5627 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 19:19:39.473486 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473484 5627 policy_none.go:50] "Start" Oct 30 19:19:39.473674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473492 5627 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 19:19:39.473674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473501 5627 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 19:19:39.473674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473638 5627 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 19:19:39.473674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473645 5627 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 19:19:39.473674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.473651 5627 policy_none.go:44] "Start" Oct 30 19:19:39.477959 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.477930 5627 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 19:19:39.480984 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.480969 5627 manager.go:341] "Starting Device Plugin manager" Oct 30 19:19:39.481052 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:19:39.480998 5627 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 19:19:39.481052 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.481011 5627 server.go:87] "Starting device plugin registration server" Oct 30 19:19:39.481755 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.481739 5627 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 19:19:39.481828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.481756 5627 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 19:19:39.481877 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.481836 5627 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 19:19:39.481972 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.481940 5627 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 19:19:39.482033 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.481985 5627 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 19:19:39.482033 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.481993 5627 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 19:19:39.482123 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.482050 5627 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:19:39.481971466Z","UUID":"23b7aeb4-56fe-4447-9aec-2a3f912951b0","Handler":null,"Name":"","Endpoint":""} Oct 30 19:19:39.483321 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:19:39.483301 5627 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 19:19:39.483588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.483568 5627 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 19:19:39.483674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.483593 5627 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 19:19:39.491062 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.491011 5627 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal"] Oct 30 19:19:39.491825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.491787 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"5c2829c8c9b99a6ca60457b5349d0af7","Type":"ContainerStarted","Data":"88606439f4f8da0cad1ab0ca31995f11d80d2e62b9dc395fe14765fe970f7def"} Oct 30 19:19:39.491931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.491839 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"5c2829c8c9b99a6ca60457b5349d0af7","Type":"ContainerDied","Data":"57540385dbe44a2ba598f090f4ff8930108a909c34ec3b54cd1202f5c5befdbf"} Oct 30 19:19:39.491931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.491856 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"5c2829c8c9b99a6ca60457b5349d0af7","Type":"ContainerStarted","Data":"7cb70437aa5333b50c7c30775d8ac6fffaefb70de87e55420f4848a0452f2027"} Oct 30 19:19:39.491931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.491859 5627 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.491931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.491868 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"1ce0953a30f3bb9b41a7629efbf90c90","Type":"ContainerStarted","Data":"a3a6f783896cd569bfe79a82b0cc9525f8ce13c5ee34812f771d1956db5e2810"} Oct 30 19:19:39.491931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.491879 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" event={"ID":"1ce0953a30f3bb9b41a7629efbf90c90","Type":"ContainerStarted","Data":"851adf46f4206e25d21c4eb5044a85e66cb1eb6fd6ed79875e712c311289df9a"} Oct 30 19:19:39.491931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.491917 5627 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.506428 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.506408 5627 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:19:39.506515 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:19:39.506466 5627 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.507465 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.507448 5627 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:19:39.507561 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:19:39.507486 5627 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.568912 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.568865 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.568912 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.568912 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.569105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.568939 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ce0953a30f3bb9b41a7629efbf90c90-config\") pod \"kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"1ce0953a30f3bb9b41a7629efbf90c90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.581980 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.581958 5627 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 19:19:39.584176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.584152 5627 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 19:19:39.584254 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.584187 5627 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 19:19:39.584254 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.584198 5627 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-211.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 30 19:19:39.584254 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.584238 5627 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.597645 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.597627 5627 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.597689 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.597682 5627 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.669971 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.669933 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.669971 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.669967 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.670177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.669988 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ce0953a30f3bb9b41a7629efbf90c90-config\") pod \"kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"1ce0953a30f3bb9b41a7629efbf90c90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.670177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.670041 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.670177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.670042 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ce0953a30f3bb9b41a7629efbf90c90-config\") pod \"kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"1ce0953a30f3bb9b41a7629efbf90c90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:39.670177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:39.670042 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c2829c8c9b99a6ca60457b5349d0af7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" (UID: \"5c2829c8c9b99a6ca60457b5349d0af7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:40.337220 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.337186 5627 apiserver.go:52] "Watching apiserver" Oct 30 19:19:40.356986 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.356962 5627 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 19:19:40.357517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.357492 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w9nxj","openshift-network-diagnostics/network-check-target-zv8fh","openshift-multus/network-metrics-daemon-gvglc","kube-system/konnectivity-agent-6n7ll","openshift-console/console-7c75bd5986-5gc79","openshift-dns/node-resolver-cgl7t","openshift-ovn-kubernetes/ovnkube-node-dvltm","openshift-ingress-canary/ingress-canary-lndpb","kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal","openshift-network-operator/iptables-alerter-m8hcp","openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8","openshift-multus/multus-additional-cni-plugins-ccmmq","kube-system/global-pull-secret-syncer-f5lqj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal","openshift-console/console-54df886c98-kcrjn","openshift-image-registry/node-ca-vvzd7","openshift-monitoring/node-exporter-zjwlv","openshift-multus/multus-95px9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls","openshift-console/console-7c74d584d8-7k5wt","openshift-dns/dns-default-79mtg","openshift-cluster-node-tuning-operator/tuned-qfrrj"] Oct 30 19:19:40.363317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.363288 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t67x5\"" Oct 30 19:19:40.363317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.363307 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 19:19:40.363504 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.363376 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ctbj8\"" Oct 30 19:19:40.363747 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.363726 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs9gl\"" Oct 30 19:19:40.363918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.363896 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vhzsf\"" Oct 30 19:19:40.363918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.363906 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rhqgv\"" Oct 30 19:19:40.364018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.363934 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 19:19:40.364018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.363995 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-dk8pw\"" Oct 30 19:19:40.365413 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.365381 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 19:19:40.366287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.366270 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6rfl7\"" Oct 30 19:19:40.366393 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.366333 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 19:19:40.370918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.370891 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-n5j86\"" Oct 30 19:19:40.371039 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371019 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vd8dh\"" Oct 30 19:19:40.371220 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371201 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gkcxf\"" Oct 30 19:19:40.371371 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371353 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 19:19:40.371588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371560 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 19:19:40.371663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371588 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-t757m\"" Oct 30 19:19:40.371663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371649 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.371663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371659 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-4vgss\"" Oct 30 19:19:40.371826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371766 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.371938 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371919 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.372007 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371935 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.372007 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.371945 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 19:19:40.372105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372044 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.372105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372047 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 19:19:40.372105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372087 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.372105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372100 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.372280 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372190 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.372280 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372192 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 19:19:40.372280 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372216 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 19:19:40.372280 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372239 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.372280 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372257 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372328 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372332 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-script-lib\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372355 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372370 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372377 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-var-lib-kubelet\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372343 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372330 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372408 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372416 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-etc-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372432 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vj9wr\"" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372449 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.372496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372488 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372564 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-config\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372617 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-system-cni-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372683 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372697 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/756b72aa-da1a-4e51-8adf-df95022277d8-agent-certs\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372748 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372753 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372754 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-host\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372821 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw4qf\" (UniqueName: \"kubernetes.io/projected/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-kube-api-access-mw4qf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372879 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.373026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.372883 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-config\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373039 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zxgfv\"" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373074 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373132 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373136 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373162 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373217 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x74g6\" (UniqueName: \"kubernetes.io/projected/eb0fad83-891c-4219-a590-846d4ca0089e-kube-api-access-x74g6\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373326 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373344 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-kubelet\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373387 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-bin\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373405 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373440 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:40.373485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373473 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q796l\" (UniqueName: \"kubernetes.io/projected/2f59ba06-3da0-4638-9cd1-df7a19645b1d-kube-api-access-q796l\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:40.373969 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373617 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.373969 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373648 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-sys\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.373969 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373683 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9994\" (UniqueName: \"kubernetes.io/projected/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-kube-api-access-k9994\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:19:40.374090 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.373999 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.374090 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374038 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-systemd-units\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.374090 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374063 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-modprobe-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.374230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374095 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-conf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.374230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374137 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-systemd\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.374230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374164 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.374230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374189 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/756b72aa-da1a-4e51-8adf-df95022277d8-agent-certs\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:19:40.374230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374193 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-systemd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.374449 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374246 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-log-socket\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.374449 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374281 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.374449 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374283 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.374449 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374345 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.374449 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374362 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ktbkj\"" Oct 30 19:19:40.374602 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374485 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-tuned\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.374637 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374611 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 19:19:40.374668 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374643 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.374668 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374652 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 19:19:40.374779 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374731 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-tmp\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.374779 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374744 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j9nbc\"" Oct 30 19:19:40.374779 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374762 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eb0fad83-891c-4219-a590-846d4ca0089e-hosts-file\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:19:40.374779 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374772 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374780 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f59ba06-3da0-4638-9cd1-df7a19645b1d-metrics-certs\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374804 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-m4g9w\"" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374819 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374838 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-tmp\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374845 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374858 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374862 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-tuned\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374847 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-cnibin\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374926 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k59h\" (UniqueName: \"kubernetes.io/projected/a489b325-efa4-4d23-b7ae-978c813aa93e-kube-api-access-8k59h\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374960 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374961 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.375015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.374986 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-netns\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375026 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-run\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375068 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb0fad83-891c-4219-a590-846d4ca0089e-tmp-dir\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375093 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375117 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-serviceca\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375125 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375148 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb0fad83-891c-4219-a590-846d4ca0089e-tmp-dir\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375151 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovn-node-metrics-cert\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375193 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rrcs\" (UniqueName: \"kubernetes.io/projected/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-kube-api-access-7rrcs\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375299 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/756b72aa-da1a-4e51-8adf-df95022277d8-konnectivity-ca\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375328 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-kubernetes\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375371 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-lib-modules\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375425 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-slash\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375471 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovn-node-metrics-cert\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375449 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-var-lib-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375492 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/756b72aa-da1a-4e51-8adf-df95022277d8-konnectivity-ca\") pod \"konnectivity-agent-6n7ll\" (UID: \"756b72aa-da1a-4e51-8adf-df95022277d8\") " pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375506 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-ovn\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375545 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysconfig\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375571 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-host\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375618 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375646 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375671 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-node-log\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375694 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-netd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375740 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-env-overrides\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375802 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-serviceca\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375822 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375854 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a489b325-efa4-4d23-b7ae-978c813aa93e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.375914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.375877 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-env-overrides\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.382728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.382693 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-ovnkube-script-lib\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.383982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.383962 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q796l\" (UniqueName: \"kubernetes.io/projected/2f59ba06-3da0-4638-9cd1-df7a19645b1d-kube-api-access-q796l\") pod \"network-metrics-daemon-gvglc\" (UID: \"2f59ba06-3da0-4638-9cd1-df7a19645b1d\") " pod="openshift-multus/network-metrics-daemon-gvglc" Oct 30 19:19:40.384173 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.384148 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9994\" (UniqueName: \"kubernetes.io/projected/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-kube-api-access-k9994\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:19:40.384482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.384466 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x74g6\" (UniqueName: \"kubernetes.io/projected/eb0fad83-891c-4219-a590-846d4ca0089e-kube-api-access-x74g6\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:19:40.384771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.384752 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw4qf\" (UniqueName: \"kubernetes.io/projected/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-kube-api-access-mw4qf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.384825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.384752 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k59h\" (UniqueName: \"kubernetes.io/projected/a489b325-efa4-4d23-b7ae-978c813aa93e-kube-api-access-8k59h\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.385289 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.385273 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rrcs\" (UniqueName: \"kubernetes.io/projected/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-kube-api-access-7rrcs\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.447360 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.447331 5627 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:40.447512 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.447443 5627 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:40.459773 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.459749 5627 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:19:40.459911 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:19:40.459800 5627 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:40.461075 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.461060 5627 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 19:19:40.464326 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.464306 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 19:19:40.464404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.464306 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-87wls\"" Oct 30 19:19:40.464404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.464391 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 19:19:40.464500 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.464401 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 19:19:40.464500 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.464438 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 19:19:40.464500 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.464454 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 19:19:40.464500 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.464494 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 19:19:40.464641 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.464617 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 19:19:40.467132 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.467117 5627 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 19:19:40.467195 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:19:40.467155 5627 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-211.us-east-2.compute.internal" Oct 30 19:19:40.467252 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.467234 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 19:19:40.476356 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476329 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysconfig\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.476445 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476369 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-host\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:19:40.476445 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476399 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6hdf\" (UniqueName: \"kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.476445 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476427 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bglv\" (UniqueName: \"kubernetes.io/projected/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-api-access-7bglv\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.476584 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476454 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.476584 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476459 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a008f757-b4b8-48a7-bb30-ffa1c1d39b75-host\") pod \"node-ca-vvzd7\" (UID: \"a008f757-b4b8-48a7-bb30-ffa1c1d39b75\") " pod="openshift-image-registry/node-ca-vvzd7" Oct 30 19:19:40.476584 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476455 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysconfig\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.476584 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476540 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.476739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476610 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.476739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476644 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-bin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.476739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476724 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-bin\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.476855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476754 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.476855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476779 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-socket-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.476855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476797 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-bin\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.476855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476837 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-sys-fs\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.476989 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476870 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.476989 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476913 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-log-socket\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.476989 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476940 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-log-socket\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.476989 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476956 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.477118 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.476993 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.477118 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477018 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-data-volume\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.477118 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477043 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477118 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477069 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.477118 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477096 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477117 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-data-volume\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477122 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477140 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477160 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477188 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-system-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477207 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477211 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477238 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.477266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477265 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-config-volume\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477290 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-tmp-dir\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477306 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-kubernetes\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477309 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477318 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477320 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-slash\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477313 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477370 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477382 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-slash\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477429 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477446 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-tmp-dir\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477446 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-node-log\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477477 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-netd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477507 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-device-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477548 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w26kz\" (UniqueName: \"kubernetes.io/projected/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kube-api-access-w26kz\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477556 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-config-volume\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477577 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-kubernetes\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477583 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477586 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477621 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477693 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-cni-netd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477725 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-node-log\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477759 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-etc-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477793 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj9kx\" (UniqueName: \"kubernetes.io/projected/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-kube-api-access-qj9kx\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477814 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-etc-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477860 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5fac3e1-506c-4ce0-b992-9108addc1e81-cert\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:40.477914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477911 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477950 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-system-cni-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477986 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-host\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.477997 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-system-cni-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478023 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-kubelet\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478038 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-host\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478067 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-wtmp\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478094 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-kubelet\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478112 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5fac3e1-506c-4ce0-b992-9108addc1e81-cert\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478142 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-dbus\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478167 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-kubelet\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478195 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac144cc0-bc7d-4430-a53b-194bcfb77145-iptables-alerter-script\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478220 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478243 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478261 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spgqv\" (UniqueName: \"kubernetes.io/projected/3f35fd1e-89aa-430a-8198-703bc598c411-kube-api-access-spgqv\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.478287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478280 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-multus\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478305 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-crio-socket\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478315 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478337 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac144cc0-bc7d-4430-a53b-194bcfb77145-iptables-alerter-script\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478358 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478411 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-modprobe-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478432 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-textfile\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478448 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-netns\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478469 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-multus-certs\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478509 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-textfile\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478510 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-host-run-netns\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478521 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac144cc0-bc7d-4430-a53b-194bcfb77145-host-slash\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478557 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-ovn\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478566 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-modprobe-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478585 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478605 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478587 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-ovn\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478688 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-kubelet-config\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:40.478740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478728 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478767 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478777 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478779 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478819 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-var-lib-kubelet\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478856 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-metrics-client-ca\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478896 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-var-lib-kubelet\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478893 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478935 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-registration-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478958 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.478976 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-sys\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479029 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-metrics-client-ca\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479017 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479042 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-systemd-units\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479062 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52f645a5-26c7-4cad-9834-71eacd7c9a88-original-pull-secret\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479081 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479085 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-systemd-units\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479095 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-sys\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479112 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479137 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479145 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479164 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-k8s-cni-cncf-io\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479189 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-hostroot\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479214 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-conf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479238 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-systemd\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479260 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ef0160-1e62-4a92-8158-bdd45eebaeab-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479272 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479304 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-systemd\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479293 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479261 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479333 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-sys\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479334 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479356 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-root\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479373 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-conf\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479389 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eb0fad83-891c-4219-a590-846d4ca0089e-hosts-file\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479410 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-cnibin\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479423 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eb0fad83-891c-4219-a590-846d4ca0089e-hosts-file\") pod \"node-resolver-cgl7t\" (UID: \"eb0fad83-891c-4219-a590-846d4ca0089e\") " pod="openshift-dns/node-resolver-cgl7t" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479444 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-tls\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.479414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479447 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a489b325-efa4-4d23-b7ae-978c813aa93e-cnibin\") pod \"multus-additional-cni-plugins-ccmmq\" (UID: \"a489b325-efa4-4d23-b7ae-978c813aa93e\") " pod="openshift-multus/multus-additional-cni-plugins-ccmmq" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479471 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-run\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479487 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-lib-modules\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479501 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-var-lib-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479517 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479533 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ef0160-1e62-4a92-8158-bdd45eebaeab-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479549 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-socket-dir-parent\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479565 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479581 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479596 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-tls\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479596 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gtcg\" (UniqueName: \"kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479621 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479638 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-netns\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479652 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-conf-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479652 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479668 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479683 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-metrics-tls\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479721 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ssdd\" (UniqueName: \"kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479736 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-var-lib-openvswitch\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479744 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-etc-selinux\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479621 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-run\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479763 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ef0160-1e62-4a92-8158-bdd45eebaeab-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479598 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-lib-modules\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479801 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479829 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cni-binary-copy\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479854 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-etc-kubernetes\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479879 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479964 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.479997 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cni-binary-copy\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480008 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480030 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480054 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-daemon-config\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480087 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-systemd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480105 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7wtf\" (UniqueName: \"kubernetes.io/projected/14ef0160-1e62-4a92-8158-bdd45eebaeab-kube-api-access-f7wtf\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480152 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-metrics-tls\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480087 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480179 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ad2d631-46c4-42d5-8c8d-a0db93319a5e-etc-sysctl-d\") pod \"tuned-qfrrj\" (UID: \"2ad2d631-46c4-42d5-8c8d-a0db93319a5e\") " pod="openshift-cluster-node-tuning-operator/tuned-qfrrj" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480203 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2hwl\" (UniqueName: \"kubernetes.io/projected/ac144cc0-bc7d-4430-a53b-194bcfb77145-kube-api-access-b2hwl\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480319 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a3d3354-c4de-4bf4-8d99-80808fe2e6f9-run-systemd\") pod \"ovnkube-node-dvltm\" (UID: \"4a3d3354-c4de-4bf4-8d99-80808fe2e6f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.480644 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481335 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-daemon-config\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481388 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cnibin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481424 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481462 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h77j\" (UniqueName: \"kubernetes.io/projected/a5fac3e1-506c-4ce0-b992-9108addc1e81-kube-api-access-5h77j\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481464 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.481483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481493 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp899\" (UniqueName: \"kubernetes.io/projected/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-kube-api-access-mp899\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:40.483378 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481558 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.483378 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481674 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.483378 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.481724 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.484312 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.484285 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bglv\" (UniqueName: \"kubernetes.io/projected/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-kube-api-access-7bglv\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.484545 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.484520 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6hdf\" (UniqueName: \"kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf\") pod \"console-7c75bd5986-5gc79\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:19:40.485376 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.485358 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w26kz\" (UniqueName: \"kubernetes.io/projected/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kube-api-access-w26kz\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.485494 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.485477 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj9kx\" (UniqueName: \"kubernetes.io/projected/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-kube-api-access-qj9kx\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.485859 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.485843 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spgqv\" (UniqueName: \"kubernetes.io/projected/3f35fd1e-89aa-430a-8198-703bc598c411-kube-api-access-spgqv\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.486783 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.486768 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz75l\" (UniqueName: \"kubernetes.io/projected/d20919c9-e2a8-4e07-89ea-bc7cc726444c-kube-api-access-zz75l\") pod \"network-check-target-zv8fh\" (UID: \"d20919c9-e2a8-4e07-89ea-bc7cc726444c\") " pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:40.486864 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.486789 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gtcg\" (UniqueName: \"kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg\") pod \"console-7c74d584d8-7k5wt\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:19:40.487391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.487375 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7wtf\" (UniqueName: \"kubernetes.io/projected/14ef0160-1e62-4a92-8158-bdd45eebaeab-kube-api-access-f7wtf\") pod \"prometheus-operator-9cd89b74c-ndcj8\" (UID: \"14ef0160-1e62-4a92-8158-bdd45eebaeab\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ndcj8" Oct 30 19:19:40.487768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.487753 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ssdd\" (UniqueName: \"kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd\") pod \"console-54df886c98-kcrjn\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:40.488611 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.488596 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2hwl\" (UniqueName: \"kubernetes.io/projected/ac144cc0-bc7d-4430-a53b-194bcfb77145-kube-api-access-b2hwl\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:19:40.488872 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.488858 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp899\" (UniqueName: \"kubernetes.io/projected/acf1ec3f-be10-4166-ba73-8994e5d8d7a1-kube-api-access-mp899\") pod \"dns-default-79mtg\" (UID: \"acf1ec3f-be10-4166-ba73-8994e5d8d7a1\") " pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:40.488968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.488955 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h77j\" (UniqueName: \"kubernetes.io/projected/a5fac3e1-506c-4ce0-b992-9108addc1e81-kube-api-access-5h77j\") pod \"ingress-canary-lndpb\" (UID: \"a5fac3e1-506c-4ce0-b992-9108addc1e81\") " pod="openshift-ingress-canary/ingress-canary-lndpb" Oct 30 19:19:40.582376 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582345 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-etc-kubernetes\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582382 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cnibin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582405 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.582499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582419 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-bin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582441 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-socket-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.582499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582464 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-sys-fs\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.582499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582466 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-cnibin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582459 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-etc-kubernetes\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582487 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-bin\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582492 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582509 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-system-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582523 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-sys-fs\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582539 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-device-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582536 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-socket-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582545 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-system-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582561 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-wtmp\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582586 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-dbus\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582590 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-device-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582612 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-kubelet\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582623 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-node-exporter-wtmp\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582642 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-multus\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582646 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-dbus\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582664 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-kubelet\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582667 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-crio-socket\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582721 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-var-lib-cni-multus\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582724 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-multus-certs\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582733 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6afdc8a2-0be6-424f-ba13-99a3fcf436dc-crio-socket\") pod \"insights-runtime-extractor-w9nxj\" (UID: \"6afdc8a2-0be6-424f-ba13-99a3fcf436dc\") " pod="openshift-insights/insights-runtime-extractor-w9nxj" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582753 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac144cc0-bc7d-4430-a53b-194bcfb77145-host-slash\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582756 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-multus-certs\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582782 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac144cc0-bc7d-4430-a53b-194bcfb77145-host-slash\") pod \"iptables-alerter-m8hcp\" (UID: \"ac144cc0-bc7d-4430-a53b-194bcfb77145\") " pod="openshift-network-operator/iptables-alerter-m8hcp" Oct 30 19:19:40.582809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582805 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-kubelet-config\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582825 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-registration-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582843 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-k8s-cni-cncf-io\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582856 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-hostroot\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582866 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52f645a5-26c7-4cad-9834-71eacd7c9a88-kubelet-config\") pod \"global-pull-secret-syncer-f5lqj\" (UID: \"52f645a5-26c7-4cad-9834-71eacd7c9a88\") " pod="kube-system/global-pull-secret-syncer-f5lqj" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582871 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-sys\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582891 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-root\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582892 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-hostroot\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582907 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-k8s-cni-cncf-io\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582911 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-socket-dir-parent\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582918 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-sys\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582930 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f35fd1e-89aa-430a-8198-703bc598c411-root\") pod \"node-exporter-zjwlv\" (UID: \"3f35fd1e-89aa-430a-8198-703bc598c411\") " pod="openshift-monitoring/node-exporter-zjwlv" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582943 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-netns\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582947 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-socket-dir-parent\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582948 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-registration-dir\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582962 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-conf-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.582988 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-etc-selinux\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.583004 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-host-run-netns\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.583012 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.583024 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-conf-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.583055 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c956bc7a-44f6-4d27-b9be-0d73f6dedd01-etc-selinux\") pod \"aws-ebs-csi-driver-node-4q6ls\" (UID: \"c956bc7a-44f6-4d27-b9be-0d73f6dedd01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4q6ls" Oct 30 19:19:40.583424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:40.583090 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcecdd80-d19c-4b8d-b38f-7067c98c7f98-multus-cni-dir\") pod \"multus-95px9\" (UID: \"dcecdd80-d19c-4b8d-b38f-7067c98c7f98\") " pod="openshift-multus/multus-95px9" Oct 30 19:19:41.020200 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:41.020164 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:41.141042 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:41.141016 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:42.048801 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:42.048766 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:42.184839 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:42.184809 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:42.293800 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:42.293770 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:42.467923 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:42.467851 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-79mtg" Oct 30 19:19:42.831524 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:42.831499 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:42.831753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:42.831740 5627 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 19:19:44.349276 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:44.349239 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:44.473806 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:44.473778 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:44.839565 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:44.839526 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:19:50.042727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:50.042682 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:19:50.493383 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:50.493340 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dvltm" Oct 30 19:19:54.675235 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:19:54.675202 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6n7ll" Oct 30 19:20:00.965405 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:00.965372 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:01.070660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:01.070627 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:05.385961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.385903 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7c74d584d8-7k5wt" podUID="eb7e1781-2624-4b67-acbe-a36a05e238a9" containerName="console" containerID="cri-o://39189adffd3531844d3be29073a790933b6ea8c74856f1952f9d3de816c9c7d3" gracePeriod=15 Oct 30 19:20:05.390541 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.390501 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7c75bd5986-5gc79" podUID="c4da4805-c7d7-408c-9f75-cd124fb938e9" containerName="console" containerID="cri-o://3a62ed38a6e0d3609ebc64798f42439e8a86887a408d63f49f961a3d6c8feefd" gracePeriod=15 Oct 30 19:20:05.523677 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.523639 5627 generic.go:356] "Generic (PLEG): container finished" podID="c4da4805-c7d7-408c-9f75-cd124fb938e9" containerID="3a62ed38a6e0d3609ebc64798f42439e8a86887a408d63f49f961a3d6c8feefd" exitCode=0 Oct 30 19:20:05.523867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.523716 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c75bd5986-5gc79" event={"ID":"c4da4805-c7d7-408c-9f75-cd124fb938e9","Type":"ContainerDied","Data":"3a62ed38a6e0d3609ebc64798f42439e8a86887a408d63f49f961a3d6c8feefd"} Oct 30 19:20:05.525583 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.525551 5627 generic.go:356] "Generic (PLEG): container finished" podID="eb7e1781-2624-4b67-acbe-a36a05e238a9" containerID="39189adffd3531844d3be29073a790933b6ea8c74856f1952f9d3de816c9c7d3" exitCode=0 Oct 30 19:20:05.525726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.525591 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c74d584d8-7k5wt" event={"ID":"eb7e1781-2624-4b67-acbe-a36a05e238a9","Type":"ContainerDied","Data":"39189adffd3531844d3be29073a790933b6ea8c74856f1952f9d3de816c9c7d3"} Oct 30 19:20:05.642118 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.642096 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:20:05.644645 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.644628 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:20:05.729400 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729373 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert\") pod \"eb7e1781-2624-4b67-acbe-a36a05e238a9\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " Oct 30 19:20:05.729538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729410 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle\") pod \"c4da4805-c7d7-408c-9f75-cd124fb938e9\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " Oct 30 19:20:05.729538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729434 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config\") pod \"c4da4805-c7d7-408c-9f75-cd124fb938e9\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " Oct 30 19:20:05.729538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729453 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert\") pod \"eb7e1781-2624-4b67-acbe-a36a05e238a9\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " Oct 30 19:20:05.729538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729481 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp\") pod \"eb7e1781-2624-4b67-acbe-a36a05e238a9\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " Oct 30 19:20:05.729538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729498 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert\") pod \"c4da4805-c7d7-408c-9f75-cd124fb938e9\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " Oct 30 19:20:05.729538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729521 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca\") pod \"eb7e1781-2624-4b67-acbe-a36a05e238a9\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729565 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config\") pod \"eb7e1781-2624-4b67-acbe-a36a05e238a9\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729597 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp\") pod \"c4da4805-c7d7-408c-9f75-cd124fb938e9\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729619 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert\") pod \"c4da4805-c7d7-408c-9f75-cd124fb938e9\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729661 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config\") pod \"eb7e1781-2624-4b67-acbe-a36a05e238a9\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729690 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config\") pod \"c4da4805-c7d7-408c-9f75-cd124fb938e9\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729755 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca\") pod \"c4da4805-c7d7-408c-9f75-cd124fb938e9\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729788 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf\" (UniqueName: \"kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf\") pod \"c4da4805-c7d7-408c-9f75-cd124fb938e9\" (UID: \"c4da4805-c7d7-408c-9f75-cd124fb938e9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729834 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg\" (UniqueName: \"kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg\") pod \"eb7e1781-2624-4b67-acbe-a36a05e238a9\" (UID: \"eb7e1781-2624-4b67-acbe-a36a05e238a9\") " Oct 30 19:20:05.729874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729860 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp" pod "eb7e1781-2624-4b67-acbe-a36a05e238a9" (UID: "eb7e1781-2624-4b67-acbe-a36a05e238a9"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:05.730322 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.729922 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle" pod "c4da4805-c7d7-408c-9f75-cd124fb938e9" (UID: "c4da4805-c7d7-408c-9f75-cd124fb938e9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:05.730322 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730105 5627 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-trusted-ca-bundle\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.730322 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730113 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca" pod "eb7e1781-2624-4b67-acbe-a36a05e238a9" (UID: "eb7e1781-2624-4b67-acbe-a36a05e238a9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:05.730322 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730130 5627 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7e1781-2624-4b67-acbe-a36a05e238a9-tmp\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.730518 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730407 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert" pod "eb7e1781-2624-4b67-acbe-a36a05e238a9" (UID: "eb7e1781-2624-4b67-acbe-a36a05e238a9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:05.730580 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730523 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config" pod "eb7e1781-2624-4b67-acbe-a36a05e238a9" (UID: "eb7e1781-2624-4b67-acbe-a36a05e238a9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:05.730978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730600 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert" pod "c4da4805-c7d7-408c-9f75-cd124fb938e9" (UID: "c4da4805-c7d7-408c-9f75-cd124fb938e9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:05.730978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730681 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp" pod "c4da4805-c7d7-408c-9f75-cd124fb938e9" (UID: "c4da4805-c7d7-408c-9f75-cd124fb938e9"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:05.730978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730946 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca" pod "c4da4805-c7d7-408c-9f75-cd124fb938e9" (UID: "c4da4805-c7d7-408c-9f75-cd124fb938e9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:05.731199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.730990 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config" pod "c4da4805-c7d7-408c-9f75-cd124fb938e9" (UID: "c4da4805-c7d7-408c-9f75-cd124fb938e9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:05.732315 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.732289 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert" pod "eb7e1781-2624-4b67-acbe-a36a05e238a9" (UID: "eb7e1781-2624-4b67-acbe-a36a05e238a9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:20:05.732504 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.732478 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert" pod "c4da4805-c7d7-408c-9f75-cd124fb938e9" (UID: "c4da4805-c7d7-408c-9f75-cd124fb938e9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:20:05.732568 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.732484 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg" pod "eb7e1781-2624-4b67-acbe-a36a05e238a9" (UID: "eb7e1781-2624-4b67-acbe-a36a05e238a9"). InnerVolumeSpecName "kube-api-access-2gtcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:05.732800 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.732776 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config" pod "c4da4805-c7d7-408c-9f75-cd124fb938e9" (UID: "c4da4805-c7d7-408c-9f75-cd124fb938e9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:20:05.732902 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.732827 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config" pod "eb7e1781-2624-4b67-acbe-a36a05e238a9" (UID: "eb7e1781-2624-4b67-acbe-a36a05e238a9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:20:05.733054 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.733037 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf" pod "c4da4805-c7d7-408c-9f75-cd124fb938e9" (UID: "c4da4805-c7d7-408c-9f75-cd124fb938e9"). InnerVolumeSpecName "kube-api-access-n6hdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:05.831116 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831094 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gtcg\" (UniqueName: \"kubernetes.io/projected/eb7e1781-2624-4b67-acbe-a36a05e238a9-kube-api-access-2gtcg\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831116 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831113 5627 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-serving-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831123 5627 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-oauth-config\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831131 5627 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-oauth-serving-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831140 5627 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-serving-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831151 5627 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831158 5627 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-oauth-config\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831167 5627 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4da4805-c7d7-408c-9f75-cd124fb938e9-tmp\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831175 5627 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-oauth-serving-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831183 5627 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7e1781-2624-4b67-acbe-a36a05e238a9-console-config\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831191 5627 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-console-config\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831199 5627 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4da4805-c7d7-408c-9f75-cd124fb938e9-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.831230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:05.831207 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6hdf\" (UniqueName: \"kubernetes.io/projected/c4da4805-c7d7-408c-9f75-cd124fb938e9-kube-api-access-n6hdf\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:20:06.530420 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.530386 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c74d584d8-7k5wt" event={"ID":"eb7e1781-2624-4b67-acbe-a36a05e238a9","Type":"ContainerDied","Data":"450f86e9d510006611076f12c8ddb5613d56d965230923dcd308b88084990b17"} Oct 30 19:20:06.530884 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.530395 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c74d584d8-7k5wt" Oct 30 19:20:06.530884 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.530442 5627 scope.go:122] "RemoveContainer" containerID="39189adffd3531844d3be29073a790933b6ea8c74856f1952f9d3de816c9c7d3" Oct 30 19:20:06.531940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.531687 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c75bd5986-5gc79" event={"ID":"c4da4805-c7d7-408c-9f75-cd124fb938e9","Type":"ContainerDied","Data":"a93f7693bcd7e5c9ee5a7d273f7618ea731544a6d881a66d04290362e78c90a2"} Oct 30 19:20:06.531940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.531735 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c75bd5986-5gc79" Oct 30 19:20:06.541143 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.541101 5627 scope.go:122] "RemoveContainer" containerID="3a62ed38a6e0d3609ebc64798f42439e8a86887a408d63f49f961a3d6c8feefd" Oct 30 19:20:06.560038 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.560020 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c74d584d8-7k5wt"] Oct 30 19:20:06.567490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.567467 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c74d584d8-7k5wt"] Oct 30 19:20:06.579719 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.579686 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c75bd5986-5gc79"] Oct 30 19:20:06.584995 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:06.584977 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c75bd5986-5gc79"] Oct 30 19:20:07.395767 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:07.395736 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4da4805-c7d7-408c-9f75-cd124fb938e9" path="/var/lib/kubelet/pods/c4da4805-c7d7-408c-9f75-cd124fb938e9/volumes" Oct 30 19:20:07.396225 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:07.396212 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb7e1781-2624-4b67-acbe-a36a05e238a9" path="/var/lib/kubelet/pods/eb7e1781-2624-4b67-acbe-a36a05e238a9/volumes" Oct 30 19:20:10.400967 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:20:10.400932 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:20:14.842306 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:14.842272 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zv8fh" Oct 30 19:20:16.600446 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:16.600410 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:16.709991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:16.709954 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:21.562673 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.562632 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc"] Oct 30 19:20:21.563074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.562881 5627 cpu_manager.go:414] "containerMap: removing container" podUID="c4da4805-c7d7-408c-9f75-cd124fb938e9" containerName="console" Oct 30 19:20:21.563074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.562891 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4da4805-c7d7-408c-9f75-cd124fb938e9" containerName="console" Oct 30 19:20:21.563074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.562899 5627 cpu_manager.go:414] "containerMap: removing container" podUID="eb7e1781-2624-4b67-acbe-a36a05e238a9" containerName="console" Oct 30 19:20:21.563074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.562904 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb7e1781-2624-4b67-acbe-a36a05e238a9" containerName="console" Oct 30 19:20:21.563074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.562943 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4da4805-c7d7-408c-9f75-cd124fb938e9" containerName="console" Oct 30 19:20:21.563074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.562949 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb7e1781-2624-4b67-acbe-a36a05e238a9" containerName="console" Oct 30 19:20:21.564647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.564622 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.570805 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.570780 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Oct 30 19:20:21.574721 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.574675 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Oct 30 19:20:21.574829 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.574762 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Oct 30 19:20:21.574829 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.574782 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Oct 30 19:20:21.574981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.574842 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Oct 30 19:20:21.574981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.574862 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Oct 30 19:20:21.574981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.574893 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v2w4b\"" Oct 30 19:20:21.577750 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.577731 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc"] Oct 30 19:20:21.629844 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.629817 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-federate-client-tls\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.629844 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.629848 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-secret-telemeter-client\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.630001 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.629879 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nltvj\" (UniqueName: \"kubernetes.io/projected/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-kube-api-access-nltvj\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.630001 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.629958 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-serving-certs-ca-bundle\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.630080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.630026 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-telemeter-client-tls\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.630080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.630044 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-metrics-client-ca\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.630080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.630061 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.630182 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.630083 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731314 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731288 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-federate-client-tls\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731321 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-secret-telemeter-client\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731361 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nltvj\" (UniqueName: \"kubernetes.io/projected/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-kube-api-access-nltvj\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731392 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-serving-certs-ca-bundle\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731431 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-telemeter-client-tls\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731631 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731453 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-metrics-client-ca\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731631 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731481 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731799 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731644 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.731799 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.731764 5627 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 19:20:21.732504 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.732454 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-metrics-client-ca\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.732603 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.732516 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-serving-certs-ca-bundle\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.732689 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.732658 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.734215 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.734191 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-secret-telemeter-client\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.734325 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.734300 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.734385 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.734316 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-federate-client-tls\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.734438 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.734427 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-telemeter-client-tls\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.741758 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.741715 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nltvj\" (UniqueName: \"kubernetes.io/projected/df041cfc-c4b4-4ee1-bc4e-c79605faaa97-kube-api-access-nltvj\") pod \"telemeter-client-dc5d5fc9c-htqfc\" (UID: \"df041cfc-c4b4-4ee1-bc4e-c79605faaa97\") " pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:21.879244 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:21.879167 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" Oct 30 19:20:22.167912 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:22.167881 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc"] Oct 30 19:20:22.170644 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:20:22.170606 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf041cfc_c4b4_4ee1_bc4e_c79605faaa97.slice/crio-b140c3e97495cd6275bad8b3c00b6d1e64bfb60c48306c00d41d92558dd33356 WatchSource:0}: Error finding container b140c3e97495cd6275bad8b3c00b6d1e64bfb60c48306c00d41d92558dd33356: Status 404 returned error can't find the container with id b140c3e97495cd6275bad8b3c00b6d1e64bfb60c48306c00d41d92558dd33356 Oct 30 19:20:22.175885 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:22.173673 5627 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:20:22.582692 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:22.582651 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" event={"ID":"df041cfc-c4b4-4ee1-bc4e-c79605faaa97","Type":"ContainerStarted","Data":"b140c3e97495cd6275bad8b3c00b6d1e64bfb60c48306c00d41d92558dd33356"} Oct 30 19:20:24.590657 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:24.590615 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" event={"ID":"df041cfc-c4b4-4ee1-bc4e-c79605faaa97","Type":"ContainerStarted","Data":"5e435f9cd3e05e560efe42ac9d6c66d1d3b1235cfa9548b420042e3a2b9649ec"} Oct 30 19:20:25.595601 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.595569 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" event={"ID":"df041cfc-c4b4-4ee1-bc4e-c79605faaa97","Type":"ContainerStarted","Data":"a34a173c0064bebdc4d91bf7313271dcecce11bc42b329cd81b3b357608f8abf"} Oct 30 19:20:25.595986 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.595621 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" event={"ID":"df041cfc-c4b4-4ee1-bc4e-c79605faaa97","Type":"ContainerStarted","Data":"7ee1f33164d1eb185d65f9b98d73f984ad5591f767118434af4c716d39f439a3"} Oct 30 19:20:25.620609 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.620564 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-dc5d5fc9c-htqfc" podStartSLOduration=2.23370067 podStartE2EDuration="4.620552041s" totalImagesPullingTime="2.386851371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:25.618496071 +0000 UTC m=+46.345313368" watchObservedRunningTime="2026-10-30 19:20:25.620552041 +0000 UTC m=+46.347369334" Oct 30 19:20:25.707054 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.707026 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-77cb5867d6-ds5kv"] Oct 30 19:20:25.709080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.709060 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.720086 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.720063 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77cb5867d6-ds5kv"] Oct 30 19:20:25.761861 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.761829 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-config\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.761861 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.761860 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-trusted-ca-bundle\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.762053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.761876 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-oauth-serving-cert\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.762053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.761895 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47wwk\" (UniqueName: \"kubernetes.io/projected/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-kube-api-access-47wwk\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.762053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.761959 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-serving-cert\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.762180 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.762110 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-service-ca\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.762180 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.762167 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-tmp\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.762252 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.762190 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-oauth-config\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.817813 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.817791 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:25.863243 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863184 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wwk\" (UniqueName: \"kubernetes.io/projected/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-kube-api-access-47wwk\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.863243 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863224 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-serving-cert\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.863392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863260 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-service-ca\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.863392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863292 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-tmp\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.863392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863321 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-oauth-config\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.863549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863466 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-config\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.863549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863508 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-trusted-ca-bundle\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.863549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863534 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-oauth-serving-cert\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.863885 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.863831 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-tmp\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.864166 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.864140 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-service-ca\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.864256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.864183 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-config\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.864382 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.864362 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-trusted-ca-bundle\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.864488 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.864466 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-oauth-serving-cert\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.865761 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.865743 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-serving-cert\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.865899 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.865883 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-oauth-config\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.873165 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.873145 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47wwk\" (UniqueName: \"kubernetes.io/projected/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-kube-api-access-47wwk\") pod \"console-77cb5867d6-ds5kv\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:25.924470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:25.924445 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:26.019317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:26.019289 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:26.288786 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:26.288754 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77cb5867d6-ds5kv"] Oct 30 19:20:26.292417 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:20:26.292387 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a1d231_2f2c_4ee7_a31a_52672d1cfaed.slice/crio-de2ec310497f89fb74e51d22535def713cf8ed0045cf94b8c598c6e70eb5ce15 WatchSource:0}: Error finding container de2ec310497f89fb74e51d22535def713cf8ed0045cf94b8c598c6e70eb5ce15: Status 404 returned error can't find the container with id de2ec310497f89fb74e51d22535def713cf8ed0045cf94b8c598c6e70eb5ce15 Oct 30 19:20:26.413312 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:26.413284 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:26.534596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:26.534569 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:26.600955 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:26.600815 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77cb5867d6-ds5kv" event={"ID":"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed","Type":"ContainerStarted","Data":"4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b"} Oct 30 19:20:26.600955 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:26.600866 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77cb5867d6-ds5kv" event={"ID":"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed","Type":"ContainerStarted","Data":"de2ec310497f89fb74e51d22535def713cf8ed0045cf94b8c598c6e70eb5ce15"} Oct 30 19:20:26.621457 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:26.621399 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-77cb5867d6-ds5kv" podStartSLOduration=1.6213808680000001 podStartE2EDuration="1.621380868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:26.61982077 +0000 UTC m=+47.346638069" watchObservedRunningTime="2026-10-30 19:20:26.621380868 +0000 UTC m=+47.348198166" Oct 30 19:20:31.787123 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:31.787086 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:31.901329 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:31.901300 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:36.019662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.019619 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:36.020188 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.019695 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:36.024273 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.024250 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:36.637272 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.637243 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:20:36.688261 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.688232 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54df886c98-kcrjn"] Oct 30 19:20:36.763021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.762991 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5ff56bc7d5-mq2fx"] Oct 30 19:20:36.765239 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.765224 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.779206 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.779183 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ff56bc7d5-mq2fx"] Oct 30 19:20:36.841131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.841101 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6660566b-b2ba-4443-b892-f58b3384db5d-console-serving-cert\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.841239 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.841137 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-trusted-ca-bundle\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.841239 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.841214 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6660566b-b2ba-4443-b892-f58b3384db5d-tmp\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.841319 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.841240 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6660566b-b2ba-4443-b892-f58b3384db5d-console-oauth-config\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.841319 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.841255 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-console-config\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.841319 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.841274 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ts44\" (UniqueName: \"kubernetes.io/projected/6660566b-b2ba-4443-b892-f58b3384db5d-kube-api-access-9ts44\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.841442 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.841375 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-oauth-serving-cert\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.841442 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.841418 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-service-ca\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.900808 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.900749 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:36.942477 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942455 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6660566b-b2ba-4443-b892-f58b3384db5d-tmp\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.942557 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942490 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6660566b-b2ba-4443-b892-f58b3384db5d-console-oauth-config\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.942557 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942511 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-console-config\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.942557 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942540 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ts44\" (UniqueName: \"kubernetes.io/projected/6660566b-b2ba-4443-b892-f58b3384db5d-kube-api-access-9ts44\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.942693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942566 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-oauth-serving-cert\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.942693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942591 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-service-ca\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.942806 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942756 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6660566b-b2ba-4443-b892-f58b3384db5d-console-serving-cert\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.942849 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942812 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-trusted-ca-bundle\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.942996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.942974 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6660566b-b2ba-4443-b892-f58b3384db5d-tmp\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.943308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.943286 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-service-ca\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.943434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.943415 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-console-config\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.943494 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.943454 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-oauth-serving-cert\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.943577 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.943562 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6660566b-b2ba-4443-b892-f58b3384db5d-trusted-ca-bundle\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.945231 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.945209 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6660566b-b2ba-4443-b892-f58b3384db5d-console-oauth-config\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.945308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.945294 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6660566b-b2ba-4443-b892-f58b3384db5d-console-serving-cert\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:36.952478 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:36.952454 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ts44\" (UniqueName: \"kubernetes.io/projected/6660566b-b2ba-4443-b892-f58b3384db5d-kube-api-access-9ts44\") pod \"console-5ff56bc7d5-mq2fx\" (UID: \"6660566b-b2ba-4443-b892-f58b3384db5d\") " pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:37.066398 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:37.066369 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:37.075330 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:37.075308 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:37.347855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:37.347737 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ff56bc7d5-mq2fx"] Oct 30 19:20:37.350832 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:20:37.350805 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6660566b_b2ba_4443_b892_f58b3384db5d.slice/crio-5c23e3144581c3aed07485a825f2d9f2bde9b59b31cea354736f82f08b133ce7 WatchSource:0}: Error finding container 5c23e3144581c3aed07485a825f2d9f2bde9b59b31cea354736f82f08b133ce7: Status 404 returned error can't find the container with id 5c23e3144581c3aed07485a825f2d9f2bde9b59b31cea354736f82f08b133ce7 Oct 30 19:20:37.526100 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:37.526066 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:37.638023 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:37.637940 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff56bc7d5-mq2fx" event={"ID":"6660566b-b2ba-4443-b892-f58b3384db5d","Type":"ContainerStarted","Data":"3a45948ab14fb7d4b65c6732ea6e525c7d03e0e19f1d8d8698c60d259309962b"} Oct 30 19:20:37.638023 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:37.637992 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff56bc7d5-mq2fx" event={"ID":"6660566b-b2ba-4443-b892-f58b3384db5d","Type":"ContainerStarted","Data":"5c23e3144581c3aed07485a825f2d9f2bde9b59b31cea354736f82f08b133ce7"} Oct 30 19:20:37.662034 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:37.661982 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5ff56bc7d5-mq2fx" podStartSLOduration=1.661968417 podStartE2EDuration="1.661968417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:37.661377445 +0000 UTC m=+58.388194744" watchObservedRunningTime="2026-10-30 19:20:37.661968417 +0000 UTC m=+58.388785713" Oct 30 19:20:37.687411 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:37.687382 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:39.285139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:39.285108 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:39.431053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:39.431021 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:40.402271 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:20:40.402236 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:20:44.401248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:44.401207 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:44.527098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:44.527068 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:47.075874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:47.075844 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:47.076299 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:47.075939 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:47.080754 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:47.080734 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:47.671276 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:47.671250 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5ff56bc7d5-mq2fx" Oct 30 19:20:47.721523 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:20:47.721484 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77cb5867d6-ds5kv"] Oct 30 19:21:00.757204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:00.757169 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:00.959158 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:00.959125 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:01.083973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:01.083894 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:01.707614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:01.707575 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-54df886c98-kcrjn" podUID="e646b5f0-ae93-4f27-a32f-78bbd45b837e" containerName="console" containerID="cri-o://551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485" gracePeriod=15 Oct 30 19:21:01.951177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:01.951129 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:21:02.015472 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015444 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert\") pod \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " Oct 30 19:21:02.015614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015478 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config\") pod \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " Oct 30 19:21:02.015614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015509 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp\") pod \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " Oct 30 19:21:02.015614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015535 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle\") pod \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " Oct 30 19:21:02.015614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015561 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert\") pod \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " Oct 30 19:21:02.015614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015579 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd\" (UniqueName: \"kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd\") pod \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " Oct 30 19:21:02.015614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015611 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca\") pod \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " Oct 30 19:21:02.015953 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015737 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config\") pod \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\" (UID: \"e646b5f0-ae93-4f27-a32f-78bbd45b837e\") " Oct 30 19:21:02.015953 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.015874 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert" pod "e646b5f0-ae93-4f27-a32f-78bbd45b837e" (UID: "e646b5f0-ae93-4f27-a32f-78bbd45b837e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:21:02.016049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.016013 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp" pod "e646b5f0-ae93-4f27-a32f-78bbd45b837e" (UID: "e646b5f0-ae93-4f27-a32f-78bbd45b837e"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:21:02.016105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.016051 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config" pod "e646b5f0-ae93-4f27-a32f-78bbd45b837e" (UID: "e646b5f0-ae93-4f27-a32f-78bbd45b837e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:21:02.016160 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.016119 5627 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-oauth-serving-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:02.016160 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.016130 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca" pod "e646b5f0-ae93-4f27-a32f-78bbd45b837e" (UID: "e646b5f0-ae93-4f27-a32f-78bbd45b837e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:21:02.016358 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.016331 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle" pod "e646b5f0-ae93-4f27-a32f-78bbd45b837e" (UID: "e646b5f0-ae93-4f27-a32f-78bbd45b837e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:21:02.018203 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.018170 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert" pod "e646b5f0-ae93-4f27-a32f-78bbd45b837e" (UID: "e646b5f0-ae93-4f27-a32f-78bbd45b837e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:21:02.018203 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.018183 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd" pod "e646b5f0-ae93-4f27-a32f-78bbd45b837e" (UID: "e646b5f0-ae93-4f27-a32f-78bbd45b837e"). InnerVolumeSpecName "kube-api-access-2ssdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:21:02.018317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.018229 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config" pod "e646b5f0-ae93-4f27-a32f-78bbd45b837e" (UID: "e646b5f0-ae93-4f27-a32f-78bbd45b837e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:21:02.116601 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.116578 5627 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:02.116601 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.116602 5627 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-oauth-config\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:02.116736 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.116613 5627 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-config\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:02.116736 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.116623 5627 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e646b5f0-ae93-4f27-a32f-78bbd45b837e-tmp\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:02.116736 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.116654 5627 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e646b5f0-ae93-4f27-a32f-78bbd45b837e-trusted-ca-bundle\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:02.116736 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.116664 5627 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e646b5f0-ae93-4f27-a32f-78bbd45b837e-console-serving-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:02.116736 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.116673 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ssdd\" (UniqueName: \"kubernetes.io/projected/e646b5f0-ae93-4f27-a32f-78bbd45b837e-kube-api-access-2ssdd\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:02.717413 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.717379 5627 generic.go:356] "Generic (PLEG): container finished" podID="e646b5f0-ae93-4f27-a32f-78bbd45b837e" containerID="551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485" exitCode=0 Oct 30 19:21:02.717580 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.717429 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54df886c98-kcrjn" Oct 30 19:21:02.717580 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.717466 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54df886c98-kcrjn" event={"ID":"e646b5f0-ae93-4f27-a32f-78bbd45b837e","Type":"ContainerDied","Data":"551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485"} Oct 30 19:21:02.717580 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.717521 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54df886c98-kcrjn" event={"ID":"e646b5f0-ae93-4f27-a32f-78bbd45b837e","Type":"ContainerDied","Data":"aa6056575f3baa76bffb23bb3f3e376a3186d92cb0f5d6814f8adaf61324c0f4"} Oct 30 19:21:02.717580 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.717555 5627 scope.go:122] "RemoveContainer" containerID="551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485" Oct 30 19:21:02.726574 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.726558 5627 scope.go:122] "RemoveContainer" containerID="551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485" Oct 30 19:21:02.726871 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:21:02.726852 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485\": container with ID starting with 551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485 not found: ID does not exist" containerID="551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485" Oct 30 19:21:02.726941 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.726879 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485"} err="failed to get container status \"551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485\": rpc error: code = NotFound desc = could not find container \"551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485\": container with ID starting with 551effd49b0a3fee2554c5c53bdcf38370465af3fc8388eb2b7e3adddc88d485 not found: ID does not exist" Oct 30 19:21:02.739426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.739407 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54df886c98-kcrjn"] Oct 30 19:21:02.746239 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:02.746221 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54df886c98-kcrjn"] Oct 30 19:21:03.396112 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:03.396077 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e646b5f0-ae93-4f27-a32f-78bbd45b837e" path="/var/lib/kubelet/pods/e646b5f0-ae93-4f27-a32f-78bbd45b837e/volumes" Oct 30 19:21:10.402546 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:21:10.402508 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:21:12.742112 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:12.742052 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-77cb5867d6-ds5kv" podUID="b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" containerName="console" containerID="cri-o://4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b" gracePeriod=15 Oct 30 19:21:12.977300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:12.977269 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:21:13.078060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.077997 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-config\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-config\") pod \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " Oct 30 19:21:13.078060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078029 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-serving-cert\") pod \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " Oct 30 19:21:13.078241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078079 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-oauth-config\") pod \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " Oct 30 19:21:13.078241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078100 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-tmp\") pod \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " Oct 30 19:21:13.078241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078117 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-kube-api-access-47wwk\" (UniqueName: \"kubernetes.io/projected/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-kube-api-access-47wwk\") pod \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " Oct 30 19:21:13.078241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078145 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-service-ca\") pod \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " Oct 30 19:21:13.078446 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078277 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-oauth-serving-cert\") pod \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " Oct 30 19:21:13.078446 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078329 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-trusted-ca-bundle\") pod \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\" (UID: \"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed\") " Oct 30 19:21:13.078543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078458 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-tmp" pod "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" (UID: "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:21:13.078543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078373 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-config" pod "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" (UID: "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:21:13.078645 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078570 5627 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-tmp\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:13.078645 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078591 5627 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-config\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:13.078748 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078682 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-service-ca" pod "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" (UID: "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:21:13.078792 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078739 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-oauth-serving-cert" pod "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" (UID: "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:21:13.078837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.078822 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-trusted-ca-bundle" pod "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" (UID: "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:21:13.080463 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.080432 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-oauth-config" pod "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" (UID: "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:21:13.080603 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.080471 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-serving-cert" pod "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" (UID: "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:21:13.080603 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.080504 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-kube-api-access-47wwk" pod "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" (UID: "b8a1d231-2f2c-4ee7-a31a-52672d1cfaed"). InnerVolumeSpecName "kube-api-access-47wwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:21:13.179637 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.179615 5627 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-oauth-config\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:13.179637 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.179636 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47wwk\" (UniqueName: \"kubernetes.io/projected/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-kube-api-access-47wwk\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:13.179794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.179648 5627 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:13.179794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.179656 5627 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-oauth-serving-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:13.179794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.179665 5627 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-trusted-ca-bundle\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:13.179794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.179674 5627 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed-console-serving-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:13.755141 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.755101 5627 generic.go:356] "Generic (PLEG): container finished" podID="b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" containerID="4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b" exitCode=0 Oct 30 19:21:13.755517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.755161 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77cb5867d6-ds5kv" Oct 30 19:21:13.755517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.755192 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77cb5867d6-ds5kv" event={"ID":"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed","Type":"ContainerDied","Data":"4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b"} Oct 30 19:21:13.755517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.755250 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77cb5867d6-ds5kv" event={"ID":"b8a1d231-2f2c-4ee7-a31a-52672d1cfaed","Type":"ContainerDied","Data":"de2ec310497f89fb74e51d22535def713cf8ed0045cf94b8c598c6e70eb5ce15"} Oct 30 19:21:13.755517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.755274 5627 scope.go:122] "RemoveContainer" containerID="4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b" Oct 30 19:21:13.763941 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.763923 5627 scope.go:122] "RemoveContainer" containerID="4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b" Oct 30 19:21:13.764286 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:21:13.764263 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b\": container with ID starting with 4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b not found: ID does not exist" containerID="4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b" Oct 30 19:21:13.764328 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.764295 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b"} err="failed to get container status \"4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b\": rpc error: code = NotFound desc = could not find container \"4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b\": container with ID starting with 4ce786cd3cfe73fb831e6c7c3609810b75909c3ae630f297d57a9f2025ddad8b not found: ID does not exist" Oct 30 19:21:13.773542 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.773511 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77cb5867d6-ds5kv"] Oct 30 19:21:13.778354 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:13.778334 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77cb5867d6-ds5kv"] Oct 30 19:21:15.395858 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:15.395818 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" path="/var/lib/kubelet/pods/b8a1d231-2f2c-4ee7-a31a-52672d1cfaed/volumes" Oct 30 19:21:29.647303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:29.647230 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:29.847058 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:29.847022 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:30.936895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.936856 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq"] Oct 30 19:21:30.937406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.937237 5627 cpu_manager.go:414] "containerMap: removing container" podUID="e646b5f0-ae93-4f27-a32f-78bbd45b837e" containerName="console" Oct 30 19:21:30.937406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.937254 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e646b5f0-ae93-4f27-a32f-78bbd45b837e" containerName="console" Oct 30 19:21:30.937406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.937273 5627 cpu_manager.go:414] "containerMap: removing container" podUID="b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" containerName="console" Oct 30 19:21:30.937406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.937279 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" containerName="console" Oct 30 19:21:30.937406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.937332 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="e646b5f0-ae93-4f27-a32f-78bbd45b837e" containerName="console" Oct 30 19:21:30.937406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.937344 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8a1d231-2f2c-4ee7-a31a-52672d1cfaed" containerName="console" Oct 30 19:21:30.940410 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.940390 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:30.946409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.946388 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Oct 30 19:21:30.946409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.946392 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Oct 30 19:21:30.950063 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.950032 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq"] Oct 30 19:21:30.992437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.992403 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmqss\" (UniqueName: \"kubernetes.io/projected/3a567b19-533b-44de-b6d9-f86dad1d3e5b-kube-api-access-nmqss\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:30.992570 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.992452 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:30.992570 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:30.992547 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:31.014612 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.014593 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-bhl57\"" Oct 30 19:21:31.093081 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.093052 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:31.093207 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.093087 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmqss\" (UniqueName: \"kubernetes.io/projected/3a567b19-533b-44de-b6d9-f86dad1d3e5b-kube-api-access-nmqss\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:31.093250 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.093229 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:31.093487 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.093471 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:31.093538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.093512 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:31.105065 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.105035 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmqss\" (UniqueName: \"kubernetes.io/projected/3a567b19-533b-44de-b6d9-f86dad1d3e5b-kube-api-access-nmqss\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:31.192830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.192759 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr"] Oct 30 19:21:31.195808 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.195792 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.204370 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.204344 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 19:21:31.204483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.204366 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 19:21:31.204483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.204393 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 30 19:21:31.204483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.204408 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 19:21:31.204483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.204401 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 30 19:21:31.204483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.204419 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 30 19:21:31.204483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.204437 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-t7289\"" Oct 30 19:21:31.215117 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.215094 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr"] Oct 30 19:21:31.250663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.250636 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:31.256618 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.256601 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:31.296244 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.296204 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.296409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.296252 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx5vl\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-kube-api-access-fx5vl\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.296409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.296289 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.296409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.296319 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.296409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.296340 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.296609 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.296421 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.296609 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.296479 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.382286 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.382255 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:31.397784 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.397762 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.397933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.397816 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx5vl\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-kube-api-access-fx5vl\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.397933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.397844 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.397933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.397861 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.397933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.397890 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.398222 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.397932 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.398222 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.397958 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.398764 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.398719 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.400262 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.400236 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.400374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.400351 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.400423 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.400375 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.400423 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.400356 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.407978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.407956 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.408069 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.407991 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx5vl\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-kube-api-access-fx5vl\") pod \"istiod-openshift-gateway-7876bfcb74-s8cqr\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.507219 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.507190 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:31.529218 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.529129 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq"] Oct 30 19:21:31.531643 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:21:31.531609 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a567b19_533b_44de_b6d9_f86dad1d3e5b.slice/crio-332d8a895732889d6d9a7d753f1584299de312af94f6da407d3a1bb56d5da08f WatchSource:0}: Error finding container 332d8a895732889d6d9a7d753f1584299de312af94f6da407d3a1bb56d5da08f: Status 404 returned error can't find the container with id 332d8a895732889d6d9a7d753f1584299de312af94f6da407d3a1bb56d5da08f Oct 30 19:21:31.782788 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.782765 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr"] Oct 30 19:21:31.784981 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:21:31.784954 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08df20a_ce1f_4aed_bcde_0eef4ebf501d.slice/crio-a54bc6eaf450c9126a15e8cce3fc298849b9ffe95cd8fca37e89ae8f0c911ad7 WatchSource:0}: Error finding container a54bc6eaf450c9126a15e8cce3fc298849b9ffe95cd8fca37e89ae8f0c911ad7: Status 404 returned error can't find the container with id a54bc6eaf450c9126a15e8cce3fc298849b9ffe95cd8fca37e89ae8f0c911ad7 Oct 30 19:21:31.809666 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.809634 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" event={"ID":"3a567b19-533b-44de-b6d9-f86dad1d3e5b","Type":"ContainerStarted","Data":"332d8a895732889d6d9a7d753f1584299de312af94f6da407d3a1bb56d5da08f"} Oct 30 19:21:31.810732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:31.810697 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" event={"ID":"d08df20a-ce1f-4aed-bcde-0eef4ebf501d","Type":"ContainerStarted","Data":"a54bc6eaf450c9126a15e8cce3fc298849b9ffe95cd8fca37e89ae8f0c911ad7"} Oct 30 19:21:32.462398 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:32.462362 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:34.641135 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:34.641098 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:21:34.641474 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:34.641180 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:21:34.828317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:34.828276 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" event={"ID":"d08df20a-ce1f-4aed-bcde-0eef4ebf501d","Type":"ContainerStarted","Data":"10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97"} Oct 30 19:21:34.828479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:34.828392 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:34.854552 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:34.854498 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" podStartSLOduration=1.001241245 podStartE2EDuration="3.854480826s" totalImagesPullingTime="2.853239581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:34.854194634 +0000 UTC m=+115.581011933" watchObservedRunningTime="2026-10-30 19:21:34.854480826 +0000 UTC m=+115.581298125" Oct 30 19:21:35.073049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:35.073003 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:35.186544 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:35.186513 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:35.835106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:35.835075 5627 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-s8cqr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:21:35.835539 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:35.835128 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" podUID="d08df20a-ce1f-4aed-bcde-0eef4ebf501d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:21:35.965636 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:35.965603 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:36.082997 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:36.082960 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:36.190867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:36.190787 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:37.843812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:37.843772 5627 generic.go:356] "Generic (PLEG): container finished" podID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerID="a49c38b9a1654ddc2dad27a292dafcdfe18887354b4e0daeab66ceddaa08a864" exitCode=0 Oct 30 19:21:37.844199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:37.843856 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" event={"ID":"3a567b19-533b-44de-b6d9-f86dad1d3e5b","Type":"ContainerDied","Data":"a49c38b9a1654ddc2dad27a292dafcdfe18887354b4e0daeab66ceddaa08a864"} Oct 30 19:21:38.547155 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:38.547113 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:38.834135 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:38.834101 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:21:40.402919 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:21:40.402880 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:21:40.855053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:40.855012 5627 generic.go:356] "Generic (PLEG): container finished" podID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerID="e3ab34722d5d3830e848e1abd55c0f2b342b25c75f377750db9b836829b48407" exitCode=0 Oct 30 19:21:40.855198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:40.855070 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" event={"ID":"3a567b19-533b-44de-b6d9-f86dad1d3e5b","Type":"ContainerDied","Data":"e3ab34722d5d3830e848e1abd55c0f2b342b25c75f377750db9b836829b48407"} Oct 30 19:21:44.442167 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:44.442130 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:46.879582 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:46.879543 5627 generic.go:356] "Generic (PLEG): container finished" podID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerID="18e4df1a4a5844542c6a41b84896c18b84bde591be5719b1db8163d2ff7b3139" exitCode=0 Oct 30 19:21:46.879963 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:46.879589 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" event={"ID":"3a567b19-533b-44de-b6d9-f86dad1d3e5b","Type":"ContainerDied","Data":"18e4df1a4a5844542c6a41b84896c18b84bde591be5719b1db8163d2ff7b3139"} Oct 30 19:21:47.999873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:47.999850 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:48.033817 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.033795 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a567b19-533b-44de-b6d9-f86dad1d3e5b-kube-api-access-nmqss\" (UniqueName: \"kubernetes.io/projected/3a567b19-533b-44de-b6d9-f86dad1d3e5b-kube-api-access-nmqss\") pod \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " Oct 30 19:21:48.033955 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.033839 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-util\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-util\") pod \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " Oct 30 19:21:48.033955 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.033865 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-bundle\") pod \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\" (UID: \"3a567b19-533b-44de-b6d9-f86dad1d3e5b\") " Oct 30 19:21:48.034237 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.034216 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-bundle" pod "3a567b19-533b-44de-b6d9-f86dad1d3e5b" (UID: "3a567b19-533b-44de-b6d9-f86dad1d3e5b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:21:48.036179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.036156 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a567b19-533b-44de-b6d9-f86dad1d3e5b-kube-api-access-nmqss" pod "3a567b19-533b-44de-b6d9-f86dad1d3e5b" (UID: "3a567b19-533b-44de-b6d9-f86dad1d3e5b"). InnerVolumeSpecName "kube-api-access-nmqss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:21:48.039399 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.039372 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-util" pod "3a567b19-533b-44de-b6d9-f86dad1d3e5b" (UID: "3a567b19-533b-44de-b6d9-f86dad1d3e5b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:21:48.135269 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.135216 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmqss\" (UniqueName: \"kubernetes.io/projected/3a567b19-533b-44de-b6d9-f86dad1d3e5b-kube-api-access-nmqss\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:48.135269 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.135242 5627 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-util\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:48.135269 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.135254 5627 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a567b19-533b-44de-b6d9-f86dad1d3e5b-bundle\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:48.888185 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.888155 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" event={"ID":"3a567b19-533b-44de-b6d9-f86dad1d3e5b","Type":"ContainerDied","Data":"332d8a895732889d6d9a7d753f1584299de312af94f6da407d3a1bb56d5da08f"} Oct 30 19:21:48.888185 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.888184 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644ng8lq" Oct 30 19:21:48.888378 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:21:48.888200 5627 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="332d8a895732889d6d9a7d753f1584299de312af94f6da407d3a1bb56d5da08f" Oct 30 19:22:02.431154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:02.431117 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:02.568132 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:02.568096 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:10.402201 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:22:10.402163 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:22:10.962833 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:10.962785 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:11.340950 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:11.340917 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:11.998134 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:11.998103 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:12.116665 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.116634 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7"] Oct 30 19:22:12.116928 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.116916 5627 cpu_manager.go:414] "containerMap: removing container" podUID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerName="util" Oct 30 19:22:12.116976 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.116930 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerName="util" Oct 30 19:22:12.116976 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.116949 5627 cpu_manager.go:414] "containerMap: removing container" podUID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerName="pull" Oct 30 19:22:12.116976 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.116954 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerName="pull" Oct 30 19:22:12.116976 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.116961 5627 cpu_manager.go:414] "containerMap: removing container" podUID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerName="extract" Oct 30 19:22:12.116976 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.116966 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerName="extract" Oct 30 19:22:12.117156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.117012 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a567b19-533b-44de-b6d9-f86dad1d3e5b" containerName="extract" Oct 30 19:22:12.122640 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.122619 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.149813 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.149748 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7"] Oct 30 19:22:12.197963 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.197939 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 30 19:22:12.197963 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.197955 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 30 19:22:12.198109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.198062 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Oct 30 19:22:12.198109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.198097 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-zpjmc\"" Oct 30 19:22:12.200457 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.200431 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Oct 30 19:22:12.200561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.200516 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Oct 30 19:22:12.203844 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.203827 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-metrics-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.203904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.203859 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.203904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.203879 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-tmp\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.203904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.203899 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.204061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.204039 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-manifests\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.204098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.204077 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf589\" (UniqueName: \"kubernetes.io/projected/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-kube-api-access-qf589\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.304775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.304720 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-manifests\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.304775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.304750 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf589\" (UniqueName: \"kubernetes.io/projected/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-kube-api-access-qf589\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.304775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.304771 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-metrics-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.304937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.304792 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.304937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.304818 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-tmp\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.304937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.304849 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.305166 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.305149 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-manifests\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.307383 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.307351 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-tmp\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.307487 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.307355 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.307487 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.307397 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.307592 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.307509 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-metrics-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.314862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.314841 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf589\" (UniqueName: \"kubernetes.io/projected/7bb0eda9-94df-4613-acba-a0ab4f0bb9a2-kube-api-access-qf589\") pod \"opendatahub-operator-controller-manager-668f68f55d-vpdg7\" (UID: \"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.316624 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.316602 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:12.434849 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.434830 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:12.446830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.446803 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:12.612780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.612695 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:12.797872 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.797846 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:12.797972 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.797898 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7"] Oct 30 19:22:12.798306 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:22:12.798287 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb0eda9_94df_4613_acba_a0ab4f0bb9a2.slice/crio-b964bb9fb2fb6291ae760b8bff782e93a0e864f81b1a3ec2e7752f75916a1277 WatchSource:0}: Error finding container b964bb9fb2fb6291ae760b8bff782e93a0e864f81b1a3ec2e7752f75916a1277: Status 404 returned error can't find the container with id b964bb9fb2fb6291ae760b8bff782e93a0e864f81b1a3ec2e7752f75916a1277 Oct 30 19:22:12.968780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:12.968664 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" event={"ID":"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2","Type":"ContainerStarted","Data":"b964bb9fb2fb6291ae760b8bff782e93a0e864f81b1a3ec2e7752f75916a1277"} Oct 30 19:22:15.013544 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:15.013490 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:15.983744 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:15.983642 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" event={"ID":"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2","Type":"ContainerStarted","Data":"8d1b36c7f299f4d09dec3a4c6cdff92e5aca5abcfdb5559ddd8be4bcc538e6e7"} Oct 30 19:22:16.158280 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:16.158250 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:16.869187 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:16.869155 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:16.988936 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:16.988901 5627 generic.go:356] "Generic (PLEG): container finished" podID="7bb0eda9-94df-4613-acba-a0ab4f0bb9a2" containerID="8d1b36c7f299f4d09dec3a4c6cdff92e5aca5abcfdb5559ddd8be4bcc538e6e7" exitCode=0 Oct 30 19:22:16.989102 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:16.988989 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" event={"ID":"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2","Type":"ContainerDied","Data":"8d1b36c7f299f4d09dec3a4c6cdff92e5aca5abcfdb5559ddd8be4bcc538e6e7"} Oct 30 19:22:17.995402 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:17.995366 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" event={"ID":"7bb0eda9-94df-4613-acba-a0ab4f0bb9a2","Type":"ContainerStarted","Data":"7008e760983342e0f4d5b2d989e65a1212b0434309c819b9a95fcf50be62c406"} Oct 30 19:22:17.995787 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:17.995536 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:18.076370 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:18.076309 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" podStartSLOduration=2.256720097 podStartE2EDuration="6.076292132s" totalImagesPullingTime="3.367226531s" totalInitContainerRuntime="452.345504ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:18.075911401 +0000 UTC m=+158.802728736" watchObservedRunningTime="2026-10-30 19:22:18.076292132 +0000 UTC m=+158.803109620" Oct 30 19:22:18.515334 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:18.515294 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:18.700690 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:18.700655 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:21.199004 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:21.198969 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:21.347337 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:21.347300 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:22.500834 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:22.500799 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:24.173304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:24.173274 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:25.434209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:25.434173 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:25.592321 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:25.592282 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:26.260419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:26.260387 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:26.476760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:26.476723 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:27.667255 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:27.667217 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:27.855131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:27.855084 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:28.163263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:28.163228 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:28.835416 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:28.835375 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:29.001498 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:29.001472 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-vpdg7" Oct 30 19:22:33.408098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:33.408054 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:34.701763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:34.701734 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:35.166411 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:35.166381 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:35.335407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:35.335381 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:35.823598 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:35.823569 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:36.740773 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:36.740747 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:36.975718 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:36.975670 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:37.393851 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.393818 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:37.648290 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.648218 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm"] Oct 30 19:22:37.653104 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.653079 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" Oct 30 19:22:37.662406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.662389 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-v7gt9\"" Oct 30 19:22:37.663980 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.663933 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Oct 30 19:22:37.664106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.664017 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Oct 30 19:22:37.665818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.665793 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm"] Oct 30 19:22:37.783514 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.783472 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73800972-7ca1-4803-bdba-641dcdb5bec9-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sw2cm\" (UID: \"73800972-7ca1-4803-bdba-641dcdb5bec9\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" Oct 30 19:22:37.783740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.783555 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h28pz\" (UniqueName: \"kubernetes.io/projected/73800972-7ca1-4803-bdba-641dcdb5bec9-kube-api-access-h28pz\") pod \"cert-manager-cainjector-5c57bbd4fc-sw2cm\" (UID: \"73800972-7ca1-4803-bdba-641dcdb5bec9\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" Oct 30 19:22:37.884661 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.884629 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h28pz\" (UniqueName: \"kubernetes.io/projected/73800972-7ca1-4803-bdba-641dcdb5bec9-kube-api-access-h28pz\") pod \"cert-manager-cainjector-5c57bbd4fc-sw2cm\" (UID: \"73800972-7ca1-4803-bdba-641dcdb5bec9\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" Oct 30 19:22:37.884881 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.884681 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73800972-7ca1-4803-bdba-641dcdb5bec9-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sw2cm\" (UID: \"73800972-7ca1-4803-bdba-641dcdb5bec9\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" Oct 30 19:22:37.900763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.900682 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73800972-7ca1-4803-bdba-641dcdb5bec9-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sw2cm\" (UID: \"73800972-7ca1-4803-bdba-641dcdb5bec9\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" Oct 30 19:22:37.901377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.901357 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h28pz\" (UniqueName: \"kubernetes.io/projected/73800972-7ca1-4803-bdba-641dcdb5bec9-kube-api-access-h28pz\") pod \"cert-manager-cainjector-5c57bbd4fc-sw2cm\" (UID: \"73800972-7ca1-4803-bdba-641dcdb5bec9\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" Oct 30 19:22:37.976019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:37.975985 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" Oct 30 19:22:38.252943 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:38.252917 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm"] Oct 30 19:22:38.256794 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:22:38.256767 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73800972_7ca1_4803_bdba_641dcdb5bec9.slice/crio-b7e1a7291df556a7d3faa777d11ab6e12036cfc2c6bdc3f8f5457b103c758393 WatchSource:0}: Error finding container b7e1a7291df556a7d3faa777d11ab6e12036cfc2c6bdc3f8f5457b103c758393: Status 404 returned error can't find the container with id b7e1a7291df556a7d3faa777d11ab6e12036cfc2c6bdc3f8f5457b103c758393 Oct 30 19:22:38.371662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:38.371619 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:39.009854 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:39.009822 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:39.064602 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:39.064568 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" event={"ID":"73800972-7ca1-4803-bdba-641dcdb5bec9","Type":"ContainerStarted","Data":"b7e1a7291df556a7d3faa777d11ab6e12036cfc2c6bdc3f8f5457b103c758393"} Oct 30 19:22:40.415312 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:22:40.415273 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:22:40.716622 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:40.716539 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:40.883437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:40.883401 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:42.077384 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:42.077308 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" event={"ID":"73800972-7ca1-4803-bdba-641dcdb5bec9","Type":"ContainerStarted","Data":"a56d083da9116938c795b11a78da9a09357a021eab1c736ff2c89b36263112b9"} Oct 30 19:22:42.096623 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:42.096558 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sw2cm" podStartSLOduration=1.53116881 podStartE2EDuration="5.096542574s" totalImagesPullingTime="3.565373764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:42.095351659 +0000 UTC m=+182.822168956" watchObservedRunningTime="2026-10-30 19:22:42.096542574 +0000 UTC m=+182.823359871" Oct 30 19:22:42.873045 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:42.873011 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:43.001246 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:43.001214 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:44.482001 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:44.481965 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:45.783554 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:45.783518 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:45.901747 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:45.901690 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:46.726407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:46.726374 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:46.897745 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:46.897690 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:46.944462 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:46.944433 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt"] Oct 30 19:22:46.952639 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:46.952618 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:46.958524 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:46.958356 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt"] Oct 30 19:22:46.961205 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:46.961184 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Oct 30 19:22:46.962068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:46.962046 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Oct 30 19:22:46.962208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:46.962140 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qrxwc\"" Oct 30 19:22:47.055831 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.055737 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkjwr\" (UniqueName: \"kubernetes.io/projected/7b4193ef-6fc1-497a-8706-ad093750a47b-kube-api-access-nkjwr\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.055969 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.055869 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b4193ef-6fc1-497a-8706-ad093750a47b-tls-certs\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.055969 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.055891 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b4193ef-6fc1-497a-8706-ad093750a47b-tmp\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.079230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.079201 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:47.157310 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.157280 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b4193ef-6fc1-497a-8706-ad093750a47b-tls-certs\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.157310 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.157313 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b4193ef-6fc1-497a-8706-ad093750a47b-tmp\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.157543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.157353 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkjwr\" (UniqueName: \"kubernetes.io/projected/7b4193ef-6fc1-497a-8706-ad093750a47b-kube-api-access-nkjwr\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.159862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.159833 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b4193ef-6fc1-497a-8706-ad093750a47b-tmp\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.160034 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.160012 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b4193ef-6fc1-497a-8706-ad093750a47b-tls-certs\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.177944 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.177913 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkjwr\" (UniqueName: \"kubernetes.io/projected/7b4193ef-6fc1-497a-8706-ad093750a47b-kube-api-access-nkjwr\") pod \"kube-auth-proxy-6944dd5f6b-8kpxt\" (UID: \"7b4193ef-6fc1-497a-8706-ad093750a47b\") " pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.226693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.226666 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:47.266821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.266794 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" Oct 30 19:22:47.370312 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.370278 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:47.557032 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:47.557003 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt"] Oct 30 19:22:47.560457 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:22:47.560430 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4193ef_6fc1_497a_8706_ad093750a47b.slice/crio-322f301904f033082b7d204788a106fa18dc5656365b9a2dd1f12c0d522546cc WatchSource:0}: Error finding container 322f301904f033082b7d204788a106fa18dc5656365b9a2dd1f12c0d522546cc: Status 404 returned error can't find the container with id 322f301904f033082b7d204788a106fa18dc5656365b9a2dd1f12c0d522546cc Oct 30 19:22:48.101052 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:48.101019 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" event={"ID":"7b4193ef-6fc1-497a-8706-ad093750a47b","Type":"ContainerStarted","Data":"322f301904f033082b7d204788a106fa18dc5656365b9a2dd1f12c0d522546cc"} Oct 30 19:22:48.480669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:48.480593 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:49.234635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:49.234579 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:50.110282 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.110244 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" event={"ID":"7b4193ef-6fc1-497a-8706-ad093750a47b","Type":"ContainerStarted","Data":"075c536be0c3ee5d121205664232a777bd65a9ed22cc065ab58ef611a11e4e60"} Oct 30 19:22:50.129519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.129468 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6944dd5f6b-8kpxt" podStartSLOduration=1.737226583 podStartE2EDuration="4.129456475s" totalImagesPullingTime="2.392229892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:50.127830566 +0000 UTC m=+190.854647864" watchObservedRunningTime="2026-10-30 19:22:50.129456475 +0000 UTC m=+190.856273766" Oct 30 19:22:50.279353 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.279318 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt"] Oct 30 19:22:50.286619 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.286598 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.287034 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.287006 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/88809f9c-f262-4f69-9a56-4876ed7404da-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.287125 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.287069 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88809f9c-f262-4f69-9a56-4876ed7404da-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.287196 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.287152 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwwts\" (UniqueName: \"kubernetes.io/projected/88809f9c-f262-4f69-9a56-4876ed7404da-kube-api-access-pwwts\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.287196 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.287186 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88809f9c-f262-4f69-9a56-4876ed7404da-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.295227 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.294354 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 30 19:22:50.295227 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.294729 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 30 19:22:50.295488 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.295468 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt"] Oct 30 19:22:50.298433 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.298413 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Oct 30 19:22:50.327230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.327203 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-hxkbk\"" Oct 30 19:22:50.387802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.387776 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88809f9c-f262-4f69-9a56-4876ed7404da-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.387895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.387822 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/88809f9c-f262-4f69-9a56-4876ed7404da-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.387895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.387856 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88809f9c-f262-4f69-9a56-4876ed7404da-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.387895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.387881 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwwts\" (UniqueName: \"kubernetes.io/projected/88809f9c-f262-4f69-9a56-4876ed7404da-kube-api-access-pwwts\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.388273 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.388250 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/88809f9c-f262-4f69-9a56-4876ed7404da-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.388340 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.388276 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88809f9c-f262-4f69-9a56-4876ed7404da-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.388835 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.388817 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88809f9c-f262-4f69-9a56-4876ed7404da-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.397381 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.397358 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwwts\" (UniqueName: \"kubernetes.io/projected/88809f9c-f262-4f69-9a56-4876ed7404da-kube-api-access-pwwts\") pod \"kserve-module-controller-manager-7566cf5fc8-zl5gt\" (UID: \"88809f9c-f262-4f69-9a56-4876ed7404da\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.600873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.600813 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:50.877825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:50.877800 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt"] Oct 30 19:22:50.880340 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:22:50.880312 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88809f9c_f262_4f69_9a56_4876ed7404da.slice/crio-37be4ffe0a12dd3f7b9873f25aa6689bf67836cbdafc6d506987f69831755dc4 WatchSource:0}: Error finding container 37be4ffe0a12dd3f7b9873f25aa6689bf67836cbdafc6d506987f69831755dc4: Status 404 returned error can't find the container with id 37be4ffe0a12dd3f7b9873f25aa6689bf67836cbdafc6d506987f69831755dc4 Oct 30 19:22:51.115007 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:51.114970 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" event={"ID":"88809f9c-f262-4f69-9a56-4876ed7404da","Type":"ContainerStarted","Data":"37be4ffe0a12dd3f7b9873f25aa6689bf67836cbdafc6d506987f69831755dc4"} Oct 30 19:22:51.555725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:51.555674 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:51.693684 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:51.693650 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:52.953978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:52.953946 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:53.124286 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:53.124253 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" event={"ID":"88809f9c-f262-4f69-9a56-4876ed7404da","Type":"ContainerStarted","Data":"261d6fa14e2eb43721dde6857ce8385171504f4e218ebd5842e9fa1ffd1ff95d"} Oct 30 19:22:53.124442 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:53.124360 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:22:53.138321 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:53.138295 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:53.147201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:53.147165 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" podStartSLOduration=1.7824104699999999 podStartE2EDuration="3.147154789s" totalImagesPullingTime="1.364744319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:53.146403156 +0000 UTC m=+193.873220452" watchObservedRunningTime="2026-10-30 19:22:53.147154789 +0000 UTC m=+193.873972080" Oct 30 19:22:56.553836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:56.553796 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:56.669229 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:22:56.669172 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:01.400340 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:01.400299 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:01.558816 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:01.558782 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:02.001365 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.001329 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:02.206365 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.206329 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:02.338066 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.337993 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:02.478905 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.478872 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:02.692075 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.692007 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c"] Oct 30 19:23:02.695375 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.695357 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.711313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.711289 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c"] Oct 30 19:23:02.711544 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.711523 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\"" Oct 30 19:23:02.711633 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.711614 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Oct 30 19:23:02.711750 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.711735 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Oct 30 19:23:02.712097 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.712077 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-s9x26\"" Oct 30 19:23:02.712097 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.712092 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\"" Oct 30 19:23:02.712308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.712292 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\"" Oct 30 19:23:02.778548 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.778523 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcpwc\" (UniqueName: \"kubernetes.io/projected/2bd40173-54c4-40cb-8c37-bf960cc72825-kube-api-access-zcpwc\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.778663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.778568 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2bd40173-54c4-40cb-8c37-bf960cc72825-manager-config\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.778731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.778695 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bd40173-54c4-40cb-8c37-bf960cc72825-cert\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.778782 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.778752 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bd40173-54c4-40cb-8c37-bf960cc72825-metrics-certs\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.879399 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.879375 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2bd40173-54c4-40cb-8c37-bf960cc72825-manager-config\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.879507 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.879433 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bd40173-54c4-40cb-8c37-bf960cc72825-cert\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.879507 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.879455 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bd40173-54c4-40cb-8c37-bf960cc72825-metrics-certs\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.879507 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.879474 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcpwc\" (UniqueName: \"kubernetes.io/projected/2bd40173-54c4-40cb-8c37-bf960cc72825-kube-api-access-zcpwc\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.880041 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.880022 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2bd40173-54c4-40cb-8c37-bf960cc72825-manager-config\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.882133 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.882109 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bd40173-54c4-40cb-8c37-bf960cc72825-cert\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.882242 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.882118 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bd40173-54c4-40cb-8c37-bf960cc72825-metrics-certs\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:02.900393 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:02.900371 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcpwc\" (UniqueName: \"kubernetes.io/projected/2bd40173-54c4-40cb-8c37-bf960cc72825-kube-api-access-zcpwc\") pod \"jobset-controller-manager-cb5bf84dc-z9v6c\" (UID: \"2bd40173-54c4-40cb-8c37-bf960cc72825\") " pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:03.006824 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:03.006796 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:03.153523 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:03.153479 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:03.289854 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:03.289795 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c"] Oct 30 19:23:03.292814 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:23:03.292787 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd40173_54c4_40cb_8c37_bf960cc72825.slice/crio-35d517000be62ec4f0670a31b8710cae7f7040781c0d643762b261f831a0c4df WatchSource:0}: Error finding container 35d517000be62ec4f0670a31b8710cae7f7040781c0d643762b261f831a0c4df: Status 404 returned error can't find the container with id 35d517000be62ec4f0670a31b8710cae7f7040781c0d643762b261f831a0c4df Oct 30 19:23:03.302745 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:03.302720 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:04.163373 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:04.163339 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" event={"ID":"2bd40173-54c4-40cb-8c37-bf960cc72825","Type":"ContainerStarted","Data":"35d517000be62ec4f0670a31b8710cae7f7040781c0d643762b261f831a0c4df"} Oct 30 19:23:07.176967 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:07.176938 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" event={"ID":"2bd40173-54c4-40cb-8c37-bf960cc72825","Type":"ContainerStarted","Data":"d3fe6150b1b5d7b17d632fc01e2faf4c1dfddacbcecaaa109e9704dd7623072b"} Oct 30 19:23:07.177325 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:07.177066 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:07.196591 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:07.196548 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" podStartSLOduration=2.18432689 podStartE2EDuration="5.196537614s" totalImagesPullingTime="3.012210724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:07.194915522 +0000 UTC m=+207.921732820" watchObservedRunningTime="2026-10-30 19:23:07.196537614 +0000 UTC m=+207.923354910" Oct 30 19:23:10.350106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:10.350071 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:10.403063 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:23:10.403024 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:23:10.474148 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:10.474115 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:14.129478 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:14.129437 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zl5gt" Oct 30 19:23:15.550795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:15.550759 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:15.676815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:15.676781 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:16.157825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:16.157789 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:17.341933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:17.341894 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:18.185396 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:18.185367 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-cb5bf84dc-z9v6c" Oct 30 19:23:19.773071 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:19.773038 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:19.901476 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:19.901446 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:20.472648 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:20.472617 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:21.348981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:21.348947 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:21.461737 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:21.461260 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:25.367328 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:25.367300 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:25.532015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:25.531977 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:25.640584 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:25.640502 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:29.750763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:29.750726 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:40.402206 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:23:40.402167 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:23:41.598937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:41.598902 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:41.855568 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:41.855493 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:42.091412 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:42.091380 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:42.370784 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:42.370748 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:42.609173 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:42.609147 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:42.762867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:42.762823 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:42.916330 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:42.916299 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:43.990481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:43.990447 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:44.125322 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:44.125290 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:44.254929 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:44.254853 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:44.561560 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:44.561492 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:46.697763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:46.697729 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:46.948000 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:46.947929 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:48.710955 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:48.710924 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr"] Oct 30 19:23:48.711440 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:48.711130 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" podUID="d08df20a-ce1f-4aed-bcde-0eef4ebf501d" containerName="discovery" containerID="cri-o://10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97" gracePeriod=30 Oct 30 19:23:48.951672 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:48.951648 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:23:49.022407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.022380 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-cacerts\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-cacerts\") pod \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " Oct 30 19:23:49.022559 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.022443 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-dns-cert\") pod \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " Oct 30 19:23:49.022559 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.022483 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-local-certs\") pod \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " Oct 30 19:23:49.022559 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.022515 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-token\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-token\") pod \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " Oct 30 19:23:49.022559 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.022531 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-kubeconfig\") pod \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " Oct 30 19:23:49.022807 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.022596 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-kube-api-access-fx5vl\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-kube-api-access-fx5vl\") pod \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " Oct 30 19:23:49.022807 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.022625 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-ca-configmap\") pod \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\" (UID: \"d08df20a-ce1f-4aed-bcde-0eef4ebf501d\") " Oct 30 19:23:49.023440 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.023363 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-ca-configmap" pod "d08df20a-ce1f-4aed-bcde-0eef4ebf501d" (UID: "d08df20a-ce1f-4aed-bcde-0eef4ebf501d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:23:49.025044 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.025013 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-local-certs" pod "d08df20a-ce1f-4aed-bcde-0eef4ebf501d" (UID: "d08df20a-ce1f-4aed-bcde-0eef4ebf501d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:23:49.025044 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.025027 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-token" pod "d08df20a-ce1f-4aed-bcde-0eef4ebf501d" (UID: "d08df20a-ce1f-4aed-bcde-0eef4ebf501d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:23:49.025207 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.025161 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-dns-cert" pod "d08df20a-ce1f-4aed-bcde-0eef4ebf501d" (UID: "d08df20a-ce1f-4aed-bcde-0eef4ebf501d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:23:49.025367 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.025337 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-kubeconfig" pod "d08df20a-ce1f-4aed-bcde-0eef4ebf501d" (UID: "d08df20a-ce1f-4aed-bcde-0eef4ebf501d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:23:49.025367 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.025342 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-kube-api-access-fx5vl" pod "d08df20a-ce1f-4aed-bcde-0eef4ebf501d" (UID: "d08df20a-ce1f-4aed-bcde-0eef4ebf501d"). InnerVolumeSpecName "kube-api-access-fx5vl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:23:49.025503 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.025346 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-cacerts" pod "d08df20a-ce1f-4aed-bcde-0eef4ebf501d" (UID: "d08df20a-ce1f-4aed-bcde-0eef4ebf501d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:23:49.093300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.093268 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck"] Oct 30 19:23:49.093647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.093631 5627 cpu_manager.go:414] "containerMap: removing container" podUID="d08df20a-ce1f-4aed-bcde-0eef4ebf501d" containerName="discovery" Oct 30 19:23:49.093765 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.093649 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d08df20a-ce1f-4aed-bcde-0eef4ebf501d" containerName="discovery" Oct 30 19:23:49.093765 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.093739 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="d08df20a-ce1f-4aed-bcde-0eef4ebf501d" containerName="discovery" Oct 30 19:23:49.096660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.096632 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.105149 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.105127 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Oct 30 19:23:49.105256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.105204 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Oct 30 19:23:49.105928 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.105907 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck"] Oct 30 19:23:49.124003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.123969 5627 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-ca-configmap\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:23:49.124003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.124005 5627 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-cacerts\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:23:49.124154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.124016 5627 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-csr-dns-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:23:49.124154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.124025 5627 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-local-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:23:49.124154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.124033 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:23:49.124154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.124041 5627 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-istio-kubeconfig\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:23:49.124154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.124049 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx5vl\" (UniqueName: \"kubernetes.io/projected/d08df20a-ce1f-4aed-bcde-0eef4ebf501d-kube-api-access-fx5vl\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:23:49.143568 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.143542 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Oct 30 19:23:49.143568 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.143561 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-hzw6m\"" Oct 30 19:23:49.183346 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.183318 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:49.224473 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.224448 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.224585 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.224490 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.224585 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.224525 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkjz4\" (UniqueName: \"kubernetes.io/projected/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-kube-api-access-fkjz4\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.224738 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.224635 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.317753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.317646 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:49.325032 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.325008 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.325158 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.325050 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.325158 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.325095 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.325158 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.325128 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkjz4\" (UniqueName: \"kubernetes.io/projected/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-kube-api-access-fkjz4\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.325914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.325890 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.326175 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.326154 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.327766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.327738 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.329024 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.329000 5627 generic.go:356] "Generic (PLEG): container finished" podID="d08df20a-ce1f-4aed-bcde-0eef4ebf501d" containerID="10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97" exitCode=0 Oct 30 19:23:49.329125 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.329053 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" Oct 30 19:23:49.329125 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.329084 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" event={"ID":"d08df20a-ce1f-4aed-bcde-0eef4ebf501d","Type":"ContainerDied","Data":"10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97"} Oct 30 19:23:49.329205 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.329153 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr" event={"ID":"d08df20a-ce1f-4aed-bcde-0eef4ebf501d","Type":"ContainerDied","Data":"a54bc6eaf450c9126a15e8cce3fc298849b9ffe95cd8fca37e89ae8f0c911ad7"} Oct 30 19:23:49.329205 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.329176 5627 scope.go:122] "RemoveContainer" containerID="10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97" Oct 30 19:23:49.337208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.337193 5627 scope.go:122] "RemoveContainer" containerID="10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97" Oct 30 19:23:49.337538 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:23:49.337515 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97\": container with ID starting with 10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97 not found: ID does not exist" containerID="10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97" Oct 30 19:23:49.337599 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.337546 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97"} err="failed to get container status \"10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97\": rpc error: code = NotFound desc = could not find container \"10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97\": container with ID starting with 10cb9cb600ac164f6919cc7b110a9a069cf952aaf6db3855ff0a5e6cc16ece97 not found: ID does not exist" Oct 30 19:23:49.340991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.340975 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkjz4\" (UniqueName: \"kubernetes.io/projected/72b21b6b-0d07-45e9-8899-b625ff8d5c5e-kube-api-access-fkjz4\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4cvck\" (UID: \"72b21b6b-0d07-45e9-8899-b625ff8d5c5e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.353926 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.353907 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr"] Oct 30 19:23:49.357971 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.357952 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-s8cqr"] Oct 30 19:23:49.396486 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.396462 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d08df20a-ce1f-4aed-bcde-0eef4ebf501d" path="/var/lib/kubelet/pods/d08df20a-ce1f-4aed-bcde-0eef4ebf501d/volumes" Oct 30 19:23:49.421397 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.421370 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:23:49.695199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:49.695133 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck"] Oct 30 19:23:49.698769 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:23:49.698739 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b21b6b_0d07_45e9_8899_b625ff8d5c5e.slice/crio-315ee10e44efe58f70f89b196f1942c25414b6d768c03b5efe3d7e37999288d8 WatchSource:0}: Error finding container 315ee10e44efe58f70f89b196f1942c25414b6d768c03b5efe3d7e37999288d8: Status 404 returned error can't find the container with id 315ee10e44efe58f70f89b196f1942c25414b6d768c03b5efe3d7e37999288d8 Oct 30 19:23:50.334669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:23:50.334629 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" event={"ID":"72b21b6b-0d07-45e9-8899-b625ff8d5c5e","Type":"ContainerStarted","Data":"315ee10e44efe58f70f89b196f1942c25414b6d768c03b5efe3d7e37999288d8"} Oct 30 19:24:01.395363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:01.395336 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:24:01.395781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:01.395385 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" event={"ID":"72b21b6b-0d07-45e9-8899-b625ff8d5c5e","Type":"ContainerStarted","Data":"76208bd1903a41b2b0d2e225656c421d17e660435f68639f05fd1f3f7c2fa825"} Oct 30 19:24:01.418913 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:01.418854 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" podStartSLOduration=0.924590003 podStartE2EDuration="12.418835628s" totalImagesPullingTime="11.494245625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:01.41649074 +0000 UTC m=+262.143308059" watchObservedRunningTime="2026-10-30 19:24:01.418835628 +0000 UTC m=+262.145652926" Oct 30 19:24:07.259089 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:07.259039 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:10.208143 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:10.208104 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:10.309348 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:10.309312 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:10.402210 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:24:10.402178 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:24:12.401498 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:12.401465 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4cvck" Oct 30 19:24:15.931752 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:15.931694 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:16.071074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:16.071039 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:17.370560 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.370523 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:17.439022 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.438992 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq"] Oct 30 19:24:17.444546 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.444530 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.455911 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.455885 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq"] Oct 30 19:24:17.460164 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.460141 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Oct 30 19:24:17.460346 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.460169 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-898sh\"" Oct 30 19:24:17.460346 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.460191 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 19:24:17.460511 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.460226 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 30 19:24:17.461576 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.461551 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-maas-api-tls\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.461672 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.461594 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clbdp\" (UniqueName: \"kubernetes.io/projected/fd6db7f5-6ca2-4daa-a618-19071c76b81f-kube-api-access-clbdp\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.461672 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.461613 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-metrics-certs\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.461826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.461753 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6db7f5-6ca2-4daa-a618-19071c76b81f-openshift-service-ca\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.463312 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.463292 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 19:24:17.547543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.547513 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:17.562783 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.562755 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clbdp\" (UniqueName: \"kubernetes.io/projected/fd6db7f5-6ca2-4daa-a618-19071c76b81f-kube-api-access-clbdp\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.562914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.562786 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-metrics-certs\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.562914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.562810 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6db7f5-6ca2-4daa-a618-19071c76b81f-openshift-service-ca\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.562914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.562858 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-maas-api-tls\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.563572 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.563553 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6db7f5-6ca2-4daa-a618-19071c76b81f-openshift-service-ca\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.565296 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.565277 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-maas-api-tls\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.565407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.565391 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-metrics-certs\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.575686 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.575663 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clbdp\" (UniqueName: \"kubernetes.io/projected/fd6db7f5-6ca2-4daa-a618-19071c76b81f-kube-api-access-clbdp\") pod \"maas-api-6d669c7fcd-7jbwq\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:17.758475 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:17.758449 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:18.034544 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:18.034470 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq"] Oct 30 19:24:18.037517 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:24:18.037487 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6db7f5_6ca2_4daa_a618_19071c76b81f.slice/crio-dcbdb5e45f2a7a978bc9e32b75960e6b67dbf3ddc2ac65d2115e1ed1caab4fd2 WatchSource:0}: Error finding container dcbdb5e45f2a7a978bc9e32b75960e6b67dbf3ddc2ac65d2115e1ed1caab4fd2: Status 404 returned error can't find the container with id dcbdb5e45f2a7a978bc9e32b75960e6b67dbf3ddc2ac65d2115e1ed1caab4fd2 Oct 30 19:24:18.453769 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:18.453671 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" event={"ID":"fd6db7f5-6ca2-4daa-a618-19071c76b81f","Type":"ContainerStarted","Data":"dcbdb5e45f2a7a978bc9e32b75960e6b67dbf3ddc2ac65d2115e1ed1caab4fd2"} Oct 30 19:24:21.468931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:21.468892 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" event={"ID":"fd6db7f5-6ca2-4daa-a618-19071c76b81f","Type":"ContainerStarted","Data":"3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb"} Oct 30 19:24:21.469306 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:21.469072 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:21.491218 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:21.491170 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" podStartSLOduration=1.896106487 podStartE2EDuration="4.491158696s" totalImagesPullingTime="2.595052209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:21.489274711 +0000 UTC m=+282.216092008" watchObservedRunningTime="2026-10-30 19:24:21.491158696 +0000 UTC m=+282.217975993" Oct 30 19:24:27.478047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:27.478020 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:27.940472 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:27.940437 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:28.062994 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.062961 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:28.198434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.198358 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dd76875b5-lmd4x"] Oct 30 19:24:28.219000 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.218967 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-lmd4x"] Oct 30 19:24:28.219146 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.219084 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.225134 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.225107 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 19:24:28.225288 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.225177 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 19:24:28.225288 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.225221 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Oct 30 19:24:28.227084 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.226768 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-plmdz\"" Oct 30 19:24:28.249824 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.249801 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-lmd4x"] Oct 30 19:24:28.249989 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:24:28.249970 5627 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-l26pc tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6dd76875b5-lmd4x" podUID="a908b9b6-ccc6-4a98-8b4f-ca750c63186f" Oct 30 19:24:28.254350 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.254329 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-tls-cert\") pod \"authorino-6dd76875b5-lmd4x\" (UID: \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\") " pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.254421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.254365 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l26pc\" (UniqueName: \"kubernetes.io/projected/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-kube-api-access-l26pc\") pod \"authorino-6dd76875b5-lmd4x\" (UID: \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\") " pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.355103 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.355069 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-tls-cert\") pod \"authorino-6dd76875b5-lmd4x\" (UID: \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\") " pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.355256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.355118 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l26pc\" (UniqueName: \"kubernetes.io/projected/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-kube-api-access-l26pc\") pod \"authorino-6dd76875b5-lmd4x\" (UID: \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\") " pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.357696 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.357677 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-tls-cert\") pod \"authorino-6dd76875b5-lmd4x\" (UID: \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\") " pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.365358 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.365336 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l26pc\" (UniqueName: \"kubernetes.io/projected/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-kube-api-access-l26pc\") pod \"authorino-6dd76875b5-lmd4x\" (UID: \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\") " pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.406495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.406464 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9"] Oct 30 19:24:28.408471 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.408456 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.422674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.422647 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9"] Oct 30 19:24:28.456300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.456227 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-maas-api-tls\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.456300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.456268 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-metrics-certs\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.456300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.456292 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbc926-da38-4920-99fc-0d073a24f109-openshift-service-ca\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.456522 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.456453 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8m8m\" (UniqueName: \"kubernetes.io/projected/a3bbc926-da38-4920-99fc-0d073a24f109-kube-api-access-r8m8m\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.495978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.495955 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.501874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.501850 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:28.557565 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.557539 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-tls-cert\" (UniqueName: \"kubernetes.io/secret/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-tls-cert\") pod \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\" (UID: \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\") " Oct 30 19:24:28.557740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.557587 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-kube-api-access-l26pc\" (UniqueName: \"kubernetes.io/projected/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-kube-api-access-l26pc\") pod \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\" (UID: \"a908b9b6-ccc6-4a98-8b4f-ca750c63186f\") " Oct 30 19:24:28.557788 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.557752 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m8m\" (UniqueName: \"kubernetes.io/projected/a3bbc926-da38-4920-99fc-0d073a24f109-kube-api-access-r8m8m\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.557840 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.557792 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-maas-api-tls\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.557840 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.557832 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-metrics-certs\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.557931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.557852 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbc926-da38-4920-99fc-0d073a24f109-openshift-service-ca\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.558904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.558876 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbc926-da38-4920-99fc-0d073a24f109-openshift-service-ca\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.560069 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.560039 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-tls-cert" pod "a908b9b6-ccc6-4a98-8b4f-ca750c63186f" (UID: "a908b9b6-ccc6-4a98-8b4f-ca750c63186f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:24:28.560438 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.560411 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-kube-api-access-l26pc" pod "a908b9b6-ccc6-4a98-8b4f-ca750c63186f" (UID: "a908b9b6-ccc6-4a98-8b4f-ca750c63186f"). InnerVolumeSpecName "kube-api-access-l26pc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:24:28.560531 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.560476 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-maas-api-tls\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.561065 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.561046 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-metrics-certs\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.574382 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.574358 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8m8m\" (UniqueName: \"kubernetes.io/projected/a3bbc926-da38-4920-99fc-0d073a24f109-kube-api-access-r8m8m\") pod \"maas-api-788b7ddf75-lfwk9\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.642776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.642737 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c5bf6b5c9-vmtjh"] Oct 30 19:24:28.644914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.644898 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:24:28.658248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.658223 5627 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-tls-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:28.658248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.658246 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l26pc\" (UniqueName: \"kubernetes.io/projected/a908b9b6-ccc6-4a98-8b4f-ca750c63186f-kube-api-access-l26pc\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:28.674739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.674697 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c5bf6b5c9-vmtjh"] Oct 30 19:24:28.719539 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.719460 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:28.759157 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.759125 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-tls-cert\") pod \"authorino-c5bf6b5c9-vmtjh\" (UID: \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\") " pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:24:28.759336 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.759186 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6vdl\" (UniqueName: \"kubernetes.io/projected/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-kube-api-access-k6vdl\") pod \"authorino-c5bf6b5c9-vmtjh\" (UID: \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\") " pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:24:28.860476 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.860441 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-tls-cert\") pod \"authorino-c5bf6b5c9-vmtjh\" (UID: \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\") " pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:24:28.860729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.860493 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6vdl\" (UniqueName: \"kubernetes.io/projected/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-kube-api-access-k6vdl\") pod \"authorino-c5bf6b5c9-vmtjh\" (UID: \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\") " pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:24:28.863502 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.863484 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-tls-cert\") pod \"authorino-c5bf6b5c9-vmtjh\" (UID: \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\") " pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:24:28.884481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.884452 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6vdl\" (UniqueName: \"kubernetes.io/projected/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-kube-api-access-k6vdl\") pod \"authorino-c5bf6b5c9-vmtjh\" (UID: \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\") " pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:24:28.954446 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.954417 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:24:28.994292 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:28.994267 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9"] Oct 30 19:24:28.996742 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:24:28.996690 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3bbc926_da38_4920_99fc_0d073a24f109.slice/crio-5edf56de8f84cde8efe800f8240c481c1ee0912d9b2b92ab6bee5ad674eb19e4 WatchSource:0}: Error finding container 5edf56de8f84cde8efe800f8240c481c1ee0912d9b2b92ab6bee5ad674eb19e4: Status 404 returned error can't find the container with id 5edf56de8f84cde8efe800f8240c481c1ee0912d9b2b92ab6bee5ad674eb19e4 Oct 30 19:24:29.227105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:29.227081 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c5bf6b5c9-vmtjh"] Oct 30 19:24:29.229612 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:24:29.229583 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0e9c4b8_53fa_4838_a446_94f3a5c525fc.slice/crio-27be66130ddc7b4582a561ea98e74827f623631794e03ad516572d96b3fd8108 WatchSource:0}: Error finding container 27be66130ddc7b4582a561ea98e74827f623631794e03ad516572d96b3fd8108: Status 404 returned error can't find the container with id 27be66130ddc7b4582a561ea98e74827f623631794e03ad516572d96b3fd8108 Oct 30 19:24:29.500947 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:29.500915 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" event={"ID":"b0e9c4b8-53fa-4838-a446-94f3a5c525fc","Type":"ContainerStarted","Data":"27be66130ddc7b4582a561ea98e74827f623631794e03ad516572d96b3fd8108"} Oct 30 19:24:29.502165 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:29.502146 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-lmd4x" Oct 30 19:24:29.502233 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:29.502183 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" event={"ID":"a3bbc926-da38-4920-99fc-0d073a24f109","Type":"ContainerStarted","Data":"5edf56de8f84cde8efe800f8240c481c1ee0912d9b2b92ab6bee5ad674eb19e4"} Oct 30 19:24:29.535640 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:29.535609 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-lmd4x"] Oct 30 19:24:29.539610 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:29.539588 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-lmd4x"] Oct 30 19:24:30.509161 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:30.509118 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" event={"ID":"a3bbc926-da38-4920-99fc-0d073a24f109","Type":"ContainerStarted","Data":"e6cfe60dfdf973ff3e7848440e24e9413ec6a67bedcbaedb0daa98a56589057e"} Oct 30 19:24:30.509622 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:30.509236 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:30.533320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:30.533237 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" podStartSLOduration=1.956225022 podStartE2EDuration="2.533222226s" totalImagesPullingTime="576.997204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:30.531380476 +0000 UTC m=+291.258197776" watchObservedRunningTime="2026-10-30 19:24:30.533222226 +0000 UTC m=+291.260039523" Oct 30 19:24:31.396696 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:31.396661 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a908b9b6-ccc6-4a98-8b4f-ca750c63186f" path="/var/lib/kubelet/pods/a908b9b6-ccc6-4a98-8b4f-ca750c63186f/volumes" Oct 30 19:24:32.519010 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:32.518973 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" event={"ID":"b0e9c4b8-53fa-4838-a446-94f3a5c525fc","Type":"ContainerStarted","Data":"fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926"} Oct 30 19:24:32.543618 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:32.543573 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" podStartSLOduration=1.896371683 podStartE2EDuration="4.543561667s" totalImagesPullingTime="2.647189984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:32.541775745 +0000 UTC m=+293.268593044" watchObservedRunningTime="2026-10-30 19:24:32.543561667 +0000 UTC m=+293.270378964" Oct 30 19:24:36.519617 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.519586 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:24:36.574833 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.574797 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq"] Oct 30 19:24:36.575046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.575022 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" podUID="fd6db7f5-6ca2-4daa-a618-19071c76b81f" containerName="maas-api" containerID="cri-o://3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb" gracePeriod=30 Oct 30 19:24:36.808035 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.808012 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:36.929946 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.929917 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd6db7f5-6ca2-4daa-a618-19071c76b81f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6db7f5-6ca2-4daa-a618-19071c76b81f-openshift-service-ca\") pod \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " Oct 30 19:24:36.930093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.929963 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-metrics-certs\") pod \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " Oct 30 19:24:36.930093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.930004 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-maas-api-tls\") pod \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " Oct 30 19:24:36.930093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.930060 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd6db7f5-6ca2-4daa-a618-19071c76b81f-kube-api-access-clbdp\" (UniqueName: \"kubernetes.io/projected/fd6db7f5-6ca2-4daa-a618-19071c76b81f-kube-api-access-clbdp\") pod \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\" (UID: \"fd6db7f5-6ca2-4daa-a618-19071c76b81f\") " Oct 30 19:24:36.930388 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.930358 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd6db7f5-6ca2-4daa-a618-19071c76b81f-openshift-service-ca" pod "fd6db7f5-6ca2-4daa-a618-19071c76b81f" (UID: "fd6db7f5-6ca2-4daa-a618-19071c76b81f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:24:36.932395 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.932366 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd6db7f5-6ca2-4daa-a618-19071c76b81f-kube-api-access-clbdp" pod "fd6db7f5-6ca2-4daa-a618-19071c76b81f" (UID: "fd6db7f5-6ca2-4daa-a618-19071c76b81f"). InnerVolumeSpecName "kube-api-access-clbdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:24:36.932487 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.932394 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-metrics-certs" pod "fd6db7f5-6ca2-4daa-a618-19071c76b81f" (UID: "fd6db7f5-6ca2-4daa-a618-19071c76b81f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:24:36.932487 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:36.932405 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-maas-api-tls" pod "fd6db7f5-6ca2-4daa-a618-19071c76b81f" (UID: "fd6db7f5-6ca2-4daa-a618-19071c76b81f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:24:37.030965 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.030928 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:37.030965 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.030958 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd6db7f5-6ca2-4daa-a618-19071c76b81f-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:37.030965 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.030968 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clbdp\" (UniqueName: \"kubernetes.io/projected/fd6db7f5-6ca2-4daa-a618-19071c76b81f-kube-api-access-clbdp\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:37.031177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.030980 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6db7f5-6ca2-4daa-a618-19071c76b81f-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:37.538131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.538098 5627 generic.go:356] "Generic (PLEG): container finished" podID="fd6db7f5-6ca2-4daa-a618-19071c76b81f" containerID="3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb" exitCode=0 Oct 30 19:24:37.538576 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.538142 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" Oct 30 19:24:37.538576 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.538151 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" event={"ID":"fd6db7f5-6ca2-4daa-a618-19071c76b81f","Type":"ContainerDied","Data":"3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb"} Oct 30 19:24:37.538576 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.538191 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq" event={"ID":"fd6db7f5-6ca2-4daa-a618-19071c76b81f","Type":"ContainerDied","Data":"dcbdb5e45f2a7a978bc9e32b75960e6b67dbf3ddc2ac65d2115e1ed1caab4fd2"} Oct 30 19:24:37.538576 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.538211 5627 scope.go:122] "RemoveContainer" containerID="3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb" Oct 30 19:24:37.546402 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.546384 5627 scope.go:122] "RemoveContainer" containerID="3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb" Oct 30 19:24:37.546662 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:24:37.546641 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb\": container with ID starting with 3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb not found: ID does not exist" containerID="3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb" Oct 30 19:24:37.546727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.546671 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb"} err="failed to get container status \"3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb\": rpc error: code = NotFound desc = could not find container \"3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb\": container with ID starting with 3f5c5bfa298429de3e54deb5740611c3285a54074dee45bb0dd76b3ec7c4abeb not found: ID does not exist" Oct 30 19:24:37.557965 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.557944 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq"] Oct 30 19:24:37.565070 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:37.565051 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-7jbwq"] Oct 30 19:24:39.397103 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:39.397068 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd6db7f5-6ca2-4daa-a618-19071c76b81f" path="/var/lib/kubelet/pods/fd6db7f5-6ca2-4daa-a618-19071c76b81f/volumes" Oct 30 19:24:39.414122 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:39.414098 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:24:39.414299 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:39.414282 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:24:40.402368 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:24:40.402332 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:24:44.593929 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:24:44.593872 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:10.403676 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:25:10.403625 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:25:27.063125 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:27.063056 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:27.184614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:27.184575 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:40.403048 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:25:40.403003 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:25:44.672763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.672722 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:44.800617 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.800591 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb"] Oct 30 19:25:44.800935 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.800923 5627 cpu_manager.go:414] "containerMap: removing container" podUID="fd6db7f5-6ca2-4daa-a618-19071c76b81f" containerName="maas-api" Oct 30 19:25:44.800980 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.800937 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd6db7f5-6ca2-4daa-a618-19071c76b81f" containerName="maas-api" Oct 30 19:25:44.801021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.800985 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd6db7f5-6ca2-4daa-a618-19071c76b81f" containerName="maas-api" Oct 30 19:25:44.803977 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.803961 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.818716 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.818682 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Oct 30 19:25:44.819023 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.819005 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Oct 30 19:25:44.820533 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.820514 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ctlhc\"" Oct 30 19:25:44.820651 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.820565 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Oct 30 19:25:44.821717 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.821681 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb"] Oct 30 19:25:44.822802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.822781 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.822905 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.822816 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.822905 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.822849 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.823021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.822904 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65b0be16-0564-48ea-85a8-48c7faa67535-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.823021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.822927 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.823021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.822996 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7md4\" (UniqueName: \"kubernetes.io/projected/65b0be16-0564-48ea-85a8-48c7faa67535-kube-api-access-q7md4\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.924363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924288 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7md4\" (UniqueName: \"kubernetes.io/projected/65b0be16-0564-48ea-85a8-48c7faa67535-kube-api-access-q7md4\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.924363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924327 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.924521 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924446 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.924521 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924503 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.924625 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924530 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65b0be16-0564-48ea-85a8-48c7faa67535-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.924625 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924550 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.924902 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924878 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.925035 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924941 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.925035 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.924962 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.926624 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.926600 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:44.926849 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.926832 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65b0be16-0564-48ea-85a8-48c7faa67535-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.927137 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.927120 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65b0be16-0564-48ea-85a8-48c7faa67535-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:44.938544 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:44.938525 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7md4\" (UniqueName: \"kubernetes.io/projected/65b0be16-0564-48ea-85a8-48c7faa67535-kube-api-access-q7md4\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cn4sb\" (UID: \"65b0be16-0564-48ea-85a8-48c7faa67535\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:45.115697 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:45.115677 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:45.404630 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:45.404488 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb"] Oct 30 19:25:45.406850 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:25:45.406818 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b0be16_0564_48ea_85a8_48c7faa67535.slice/crio-953bfbf40e3f7968f4438a24655b527890d0ff452f7f6e4d3c62d7b827578bee WatchSource:0}: Error finding container 953bfbf40e3f7968f4438a24655b527890d0ff452f7f6e4d3c62d7b827578bee: Status 404 returned error can't find the container with id 953bfbf40e3f7968f4438a24655b527890d0ff452f7f6e4d3c62d7b827578bee Oct 30 19:25:45.408101 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:45.408085 5627 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:25:45.795481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:45.795438 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" event={"ID":"65b0be16-0564-48ea-85a8-48c7faa67535","Type":"ContainerStarted","Data":"953bfbf40e3f7968f4438a24655b527890d0ff452f7f6e4d3c62d7b827578bee"} Oct 30 19:25:46.158153 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.157331 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q"] Oct 30 19:25:46.175458 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.175420 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q"] Oct 30 19:25:46.175631 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.175526 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.188504 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.188458 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Oct 30 19:25:46.237206 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.237162 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.237385 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.237224 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.237385 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.237256 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.237385 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.237280 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d4749bf-c669-4371-af0f-e675731c3fc2-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.237385 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.237322 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.237385 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.237357 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh7ml\" (UniqueName: \"kubernetes.io/projected/2d4749bf-c669-4371-af0f-e675731c3fc2-kube-api-access-zh7ml\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.250721 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.250676 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:46.338812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.338766 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.338996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.338838 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh7ml\" (UniqueName: \"kubernetes.io/projected/2d4749bf-c669-4371-af0f-e675731c3fc2-kube-api-access-zh7ml\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.338996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.338922 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.338996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.338955 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.338996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.338979 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.339228 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.338999 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d4749bf-c669-4371-af0f-e675731c3fc2-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.339806 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.339783 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.340213 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.340192 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.340281 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.340262 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.342171 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.342140 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d4749bf-c669-4371-af0f-e675731c3fc2-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.343026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.343004 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d4749bf-c669-4371-af0f-e675731c3fc2-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.353650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.353621 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh7ml\" (UniqueName: \"kubernetes.io/projected/2d4749bf-c669-4371-af0f-e675731c3fc2-kube-api-access-zh7ml\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bj75q\" (UID: \"2d4749bf-c669-4371-af0f-e675731c3fc2\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.492377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.492335 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:46.787313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.787278 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q"] Oct 30 19:25:46.790281 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:25:46.790252 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4749bf_c669_4371_af0f_e675731c3fc2.slice/crio-672e134fce90eebd5a70a6c95ecf033a7a930bd6c17a72cce469031d6950a907 WatchSource:0}: Error finding container 672e134fce90eebd5a70a6c95ecf033a7a930bd6c17a72cce469031d6950a907: Status 404 returned error can't find the container with id 672e134fce90eebd5a70a6c95ecf033a7a930bd6c17a72cce469031d6950a907 Oct 30 19:25:46.801027 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:46.800958 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" event={"ID":"2d4749bf-c669-4371-af0f-e675731c3fc2","Type":"ContainerStarted","Data":"672e134fce90eebd5a70a6c95ecf033a7a930bd6c17a72cce469031d6950a907"} Oct 30 19:25:47.040894 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.040865 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw"] Oct 30 19:25:47.045497 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.045472 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.055482 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.055418 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Oct 30 19:25:47.064872 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.064842 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw"] Oct 30 19:25:47.145772 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.145747 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21a18edf-99e1-4344-b1e8-0e361f5d2c24-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.145898 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.145778 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.145898 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.145794 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cmt4\" (UniqueName: \"kubernetes.io/projected/21a18edf-99e1-4344-b1e8-0e361f5d2c24-kube-api-access-2cmt4\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.146042 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.146018 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.146100 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.146075 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.146158 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.146119 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.147003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.146974 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:47.246919 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.246888 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21a18edf-99e1-4344-b1e8-0e361f5d2c24-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.246919 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.246921 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.246919 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.246940 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cmt4\" (UniqueName: \"kubernetes.io/projected/21a18edf-99e1-4344-b1e8-0e361f5d2c24-kube-api-access-2cmt4\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.247250 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.247089 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.247250 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.247148 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.247250 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.247180 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.247427 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.247372 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.247548 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.247526 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.247548 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.247539 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.249374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.249345 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21a18edf-99e1-4344-b1e8-0e361f5d2c24-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.249895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.249873 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21a18edf-99e1-4344-b1e8-0e361f5d2c24-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.263871 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.263845 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cmt4\" (UniqueName: \"kubernetes.io/projected/21a18edf-99e1-4344-b1e8-0e361f5d2c24-kube-api-access-2cmt4\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-4qmpw\" (UID: \"21a18edf-99e1-4344-b1e8-0e361f5d2c24\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.362066 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.361963 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:47.636400 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.636208 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw"] Oct 30 19:25:47.638605 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:25:47.638579 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a18edf_99e1_4344_b1e8_0e361f5d2c24.slice/crio-6b888d4cc8ad2e942768c041a1af262ecd080a078890644effd959dbf1a27d34 WatchSource:0}: Error finding container 6b888d4cc8ad2e942768c041a1af262ecd080a078890644effd959dbf1a27d34: Status 404 returned error can't find the container with id 6b888d4cc8ad2e942768c041a1af262ecd080a078890644effd959dbf1a27d34 Oct 30 19:25:47.806501 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.806455 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" event={"ID":"65b0be16-0564-48ea-85a8-48c7faa67535","Type":"ContainerStarted","Data":"12962b3db8c5ea8de3e8dec75cdd7093f0c942ce7c7c7e157b813d937e45e1c0"} Oct 30 19:25:47.807686 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.807662 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" event={"ID":"21a18edf-99e1-4344-b1e8-0e361f5d2c24","Type":"ContainerStarted","Data":"6b888d4cc8ad2e942768c041a1af262ecd080a078890644effd959dbf1a27d34"} Oct 30 19:25:47.809133 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.809106 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" event={"ID":"2d4749bf-c669-4371-af0f-e675731c3fc2","Type":"ContainerStarted","Data":"1df71aa91660d09bd46003630373fb91be15e5fdbc69839999c637c395a9d6b9"} Oct 30 19:25:47.825589 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.825546 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" podStartSLOduration=2.245092567 podStartE2EDuration="3.825533471s" totalImagesPullingTime="1.580440904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:47.824573851 +0000 UTC m=+368.551391149" watchObservedRunningTime="2026-10-30 19:25:47.825533471 +0000 UTC m=+368.552350778" Oct 30 19:25:47.849258 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:47.849208 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" podStartSLOduration=1.5584833439999999 podStartE2EDuration="1.849189596s" totalImagesPullingTime="290.706252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:47.846941921 +0000 UTC m=+368.573759218" watchObservedRunningTime="2026-10-30 19:25:47.849189596 +0000 UTC m=+368.576006893" Oct 30 19:25:48.116364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:48.116325 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:48.493271 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:48.493241 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:48.814940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:48.814852 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" event={"ID":"21a18edf-99e1-4344-b1e8-0e361f5d2c24","Type":"ContainerStarted","Data":"d1df82aa231897ea334546edb6773833c3a636b217a55756689b5bd569909180"} Oct 30 19:25:48.834996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:48.834949 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" podStartSLOduration=1.5610943659999998 podStartE2EDuration="1.834936245s" totalImagesPullingTime="273.841879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:48.833633898 +0000 UTC m=+369.560451194" watchObservedRunningTime="2026-10-30 19:25:48.834936245 +0000 UTC m=+369.561753542" Oct 30 19:25:48.997476 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:48.997444 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp"] Oct 30 19:25:49.000697 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.000677 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.012305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.012279 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp"] Oct 30 19:25:49.012431 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.012416 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Oct 30 19:25:49.059350 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.059331 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:49.064447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.064418 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.064556 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.064497 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc884\" (UniqueName: \"kubernetes.io/projected/93c33d89-f6fc-406f-928d-4819b28bbf73-kube-api-access-qc884\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.064608 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.064558 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.064608 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.064579 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.064686 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.064651 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.064738 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.064722 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93c33d89-f6fc-406f-928d-4819b28bbf73-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165090 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165038 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc884\" (UniqueName: \"kubernetes.io/projected/93c33d89-f6fc-406f-928d-4819b28bbf73-kube-api-access-qc884\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165084 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165111 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165133 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165166 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93c33d89-f6fc-406f-928d-4819b28bbf73-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165410 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165209 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165474 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165500 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.165584 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.165545 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.167569 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.167545 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93c33d89-f6fc-406f-928d-4819b28bbf73-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.167803 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.167788 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93c33d89-f6fc-406f-928d-4819b28bbf73-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.175216 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.175197 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc884\" (UniqueName: \"kubernetes.io/projected/93c33d89-f6fc-406f-928d-4819b28bbf73-kube-api-access-qc884\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp\" (UID: \"93c33d89-f6fc-406f-928d-4819b28bbf73\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.313396 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.313373 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:49.369832 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.367861 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:49.590635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.590613 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp"] Oct 30 19:25:49.592503 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:25:49.592479 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c33d89_f6fc_406f_928d_4819b28bbf73.slice/crio-c7af55f60a5e3eb73575f729987b19683f2854c0ac53dcca45df4385b65fc4e5 WatchSource:0}: Error finding container c7af55f60a5e3eb73575f729987b19683f2854c0ac53dcca45df4385b65fc4e5: Status 404 returned error can't find the container with id c7af55f60a5e3eb73575f729987b19683f2854c0ac53dcca45df4385b65fc4e5 Oct 30 19:25:49.736395 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.736363 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-lvm6p"] Oct 30 19:25:49.741030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.741006 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.752259 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.752235 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-lvm6p"] Oct 30 19:25:49.752903 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.752877 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\"" Oct 30 19:25:49.770934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.770908 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.771046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.770941 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.771046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.770966 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8648c2-b866-44c0-8371-e9223a8741eb-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.771141 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.771117 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-home\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.771236 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.771216 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.771294 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.771264 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96h2j\" (UniqueName: \"kubernetes.io/projected/5a8648c2-b866-44c0-8371-e9223a8741eb-kube-api-access-96h2j\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.812098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.812057 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:49.820609 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.820573 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" event={"ID":"93c33d89-f6fc-406f-928d-4819b28bbf73","Type":"ContainerStarted","Data":"c7af55f60a5e3eb73575f729987b19683f2854c0ac53dcca45df4385b65fc4e5"} Oct 30 19:25:49.872588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.872513 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.872588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.872565 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96h2j\" (UniqueName: \"kubernetes.io/projected/5a8648c2-b866-44c0-8371-e9223a8741eb-kube-api-access-96h2j\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.872838 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.872671 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.872838 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.872756 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.872838 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.872790 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8648c2-b866-44c0-8371-e9223a8741eb-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.873012 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.872854 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-home\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.873483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.873336 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.873483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.873349 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.873483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.873443 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-home\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.875550 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.875531 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8648c2-b866-44c0-8371-e9223a8741eb-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.875904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.875883 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8648c2-b866-44c0-8371-e9223a8741eb-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:49.885683 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:49.885658 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96h2j\" (UniqueName: \"kubernetes.io/projected/5a8648c2-b866-44c0-8371-e9223a8741eb-kube-api-access-96h2j\") pod \"echo-simulator-kserve-64b6f8f96c-lvm6p\" (UID: \"5a8648c2-b866-44c0-8371-e9223a8741eb\") " pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:50.057921 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:50.057890 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:50.334826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:50.334773 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-lvm6p"] Oct 30 19:25:50.338531 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:25:50.338496 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8648c2_b866_44c0_8371_e9223a8741eb.slice/crio-00806c5b8bbf01d6847177541337db5448644e0b0297b6a26566d68fc3efea48 WatchSource:0}: Error finding container 00806c5b8bbf01d6847177541337db5448644e0b0297b6a26566d68fc3efea48: Status 404 returned error can't find the container with id 00806c5b8bbf01d6847177541337db5448644e0b0297b6a26566d68fc3efea48 Oct 30 19:25:50.827024 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:50.826962 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" event={"ID":"93c33d89-f6fc-406f-928d-4819b28bbf73","Type":"ContainerStarted","Data":"95e6c1bd2e7376713ef4be7b8f06fc1ed94f84515107e5a3ffe9524911e76bcc"} Oct 30 19:25:50.828308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:50.828281 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" event={"ID":"5a8648c2-b866-44c0-8371-e9223a8741eb","Type":"ContainerStarted","Data":"00806c5b8bbf01d6847177541337db5448644e0b0297b6a26566d68fc3efea48"} Oct 30 19:25:50.849768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:50.849695 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" podStartSLOduration=2.549887646 podStartE2EDuration="2.849680042s" totalImagesPullingTime="299.792396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:50.849294755 +0000 UTC m=+371.576112054" watchObservedRunningTime="2026-10-30 19:25:50.849680042 +0000 UTC m=+371.576497338" Oct 30 19:25:51.225964 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.225877 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj"] Oct 30 19:25:51.231923 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.231893 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.241616 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.241580 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Oct 30 19:25:51.248892 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.248863 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj"] Oct 30 19:25:51.300726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.299145 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbz4q\" (UniqueName: \"kubernetes.io/projected/8282cab2-7b52-4403-ac9e-a9897438dfba-kube-api-access-nbz4q\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.300726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.299202 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.300726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.299239 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.300726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.299280 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8282cab2-7b52-4403-ac9e-a9897438dfba-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.300726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.299340 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.300726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.299369 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.315727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.313779 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:51.380985 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.380950 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:51.400907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.400283 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbz4q\" (UniqueName: \"kubernetes.io/projected/8282cab2-7b52-4403-ac9e-a9897438dfba-kube-api-access-nbz4q\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.400907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.400332 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.400907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.400371 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.400907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.400417 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8282cab2-7b52-4403-ac9e-a9897438dfba-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.400907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.400480 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.400907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.400507 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.401349 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.401080 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.401349 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.401126 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.401349 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.401331 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.404191 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.404161 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8282cab2-7b52-4403-ac9e-a9897438dfba-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.409405 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.409351 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8282cab2-7b52-4403-ac9e-a9897438dfba-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.413391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.413360 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbz4q\" (UniqueName: \"kubernetes.io/projected/8282cab2-7b52-4403-ac9e-a9897438dfba-kube-api-access-nbz4q\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj\" (UID: \"8282cab2-7b52-4403-ac9e-a9897438dfba\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.551353 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.551318 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:51.847626 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:51.847598 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj"] Oct 30 19:25:51.849075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:25:51.849050 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8282cab2_7b52_4403_ac9e_a9897438dfba.slice/crio-bdaf91e0cf671a654e5ebb9a76dc9a5539036380b86b4e33870d1261da46d2da WatchSource:0}: Error finding container bdaf91e0cf671a654e5ebb9a76dc9a5539036380b86b4e33870d1261da46d2da: Status 404 returned error can't find the container with id bdaf91e0cf671a654e5ebb9a76dc9a5539036380b86b4e33870d1261da46d2da Oct 30 19:25:52.839230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:52.839190 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" event={"ID":"5a8648c2-b866-44c0-8371-e9223a8741eb","Type":"ContainerStarted","Data":"89859718e50ed531b19cdb6e5eb04cbad85a08bf3b0306e789dbcbe02e1acfaf"} Oct 30 19:25:52.841029 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:52.840998 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" event={"ID":"8282cab2-7b52-4403-ac9e-a9897438dfba","Type":"ContainerStarted","Data":"54be0c39838fd33782293f3a68f5e7988c40accd6b3a603c4da272d552a1c4b5"} Oct 30 19:25:52.841141 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:52.841045 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" event={"ID":"8282cab2-7b52-4403-ac9e-a9897438dfba","Type":"ContainerStarted","Data":"bdaf91e0cf671a654e5ebb9a76dc9a5539036380b86b4e33870d1261da46d2da"} Oct 30 19:25:52.861438 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:52.861392 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" podStartSLOduration=1.979369371 podStartE2EDuration="3.86137898s" totalImagesPullingTime="1.882009609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:52.8606956 +0000 UTC m=+373.587512897" watchObservedRunningTime="2026-10-30 19:25:52.86137898 +0000 UTC m=+373.588196276" Oct 30 19:25:52.885693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:52.885639 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" podStartSLOduration=1.591509536 podStartE2EDuration="1.885619811s" totalImagesPullingTime="294.110275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:52.883243657 +0000 UTC m=+373.610060958" watchObservedRunningTime="2026-10-30 19:25:52.885619811 +0000 UTC m=+373.612437109" Oct 30 19:25:53.058545 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:53.058507 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:25:53.551618 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:53.551590 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:25:55.116093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:55.116053 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:55.128848 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:55.128824 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:55.864752 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:55.864724 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cn4sb" Oct 30 19:25:55.974815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:55.974782 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm"] Oct 30 19:25:55.978283 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:55.978262 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:55.987287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:55.987250 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Oct 30 19:25:55.988406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:55.988386 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm"] Oct 30 19:25:56.044308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.044280 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.044308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.044307 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.044463 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.044334 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.044497 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.044461 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eb64bc3-1b22-4944-b693-be0833371aaf-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.044497 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.044482 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.044598 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.044510 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlzt7\" (UniqueName: \"kubernetes.io/projected/7eb64bc3-1b22-4944-b693-be0833371aaf-kube-api-access-tlzt7\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.054402 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.054360 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:56.145874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.145817 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.145874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.145845 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.146245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.145875 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.146245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.145930 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eb64bc3-1b22-4944-b693-be0833371aaf-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.146245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.145964 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.146245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.146007 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlzt7\" (UniqueName: \"kubernetes.io/projected/7eb64bc3-1b22-4944-b693-be0833371aaf-kube-api-access-tlzt7\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.146394 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.146276 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.146394 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.146306 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.146464 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.146406 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.148336 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.148318 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eb64bc3-1b22-4944-b693-be0833371aaf-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.148558 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.148540 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eb64bc3-1b22-4944-b693-be0833371aaf-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.156212 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.156194 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlzt7\" (UniqueName: \"kubernetes.io/projected/7eb64bc3-1b22-4944-b693-be0833371aaf-kube-api-access-tlzt7\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm\" (UID: \"7eb64bc3-1b22-4944-b693-be0833371aaf\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.290316 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.290295 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:56.493036 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.493002 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:56.505492 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.505465 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:56.564959 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.564934 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm"] Oct 30 19:25:56.566828 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:25:56.566804 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7eb64bc3_1b22_4944_b693_be0833371aaf.slice/crio-fedfc5589dc6276481f7ad0d4299bca6a4f6c3a378661e4aa7c483269de8ad41 WatchSource:0}: Error finding container fedfc5589dc6276481f7ad0d4299bca6a4f6c3a378661e4aa7c483269de8ad41: Status 404 returned error can't find the container with id fedfc5589dc6276481f7ad0d4299bca6a4f6c3a378661e4aa7c483269de8ad41 Oct 30 19:25:56.858573 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.858543 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" event={"ID":"7eb64bc3-1b22-4944-b693-be0833371aaf","Type":"ContainerStarted","Data":"fedfc5589dc6276481f7ad0d4299bca6a4f6c3a378661e4aa7c483269de8ad41"} Oct 30 19:25:56.875569 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:56.875540 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" Oct 30 19:25:57.220928 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:57.220843 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:57.362721 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:57.362678 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:57.376319 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:57.376287 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:57.864415 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:57.864377 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" event={"ID":"7eb64bc3-1b22-4944-b693-be0833371aaf","Type":"ContainerStarted","Data":"de4490c07489bf530fae6a45939289f49ef90da469174629fe07eb9b2928f6b5"} Oct 30 19:25:57.876019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:57.875996 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-4qmpw" Oct 30 19:25:57.888992 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:57.888950 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podStartSLOduration=2.599317991 podStartE2EDuration="2.888938443s" totalImagesPullingTime="289.620452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:57.886811365 +0000 UTC m=+378.613628663" watchObservedRunningTime="2026-10-30 19:25:57.888938443 +0000 UTC m=+378.615755851" Oct 30 19:25:58.290660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:58.290616 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:25:59.314198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:59.314161 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:59.327775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:59.327740 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:25:59.885087 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:25:59.885054 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp" Oct 30 19:26:00.058467 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:00.058430 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:26:00.070958 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:00.070931 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:26:00.890246 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:00.890213 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" Oct 30 19:26:01.552400 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:01.552370 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:26:01.564655 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:01.564626 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:26:01.892924 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:01.892854 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" Oct 30 19:26:06.290821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:06.290783 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:26:06.303307 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:06.303282 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:26:06.914327 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:06.914297 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:26:10.269664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.269625 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c87c8b988-knzjq"] Oct 30 19:26:10.273873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.273853 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c87c8b988-knzjq" Oct 30 19:26:10.282750 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.282725 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c87c8b988-knzjq"] Oct 30 19:26:10.378108 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.378081 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/408b1443-a9e2-493d-9597-1e5512dd88ff-tls-cert\") pod \"authorino-c87c8b988-knzjq\" (UID: \"408b1443-a9e2-493d-9597-1e5512dd88ff\") " pod="kuadrant-system/authorino-c87c8b988-knzjq" Oct 30 19:26:10.378274 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.378147 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gqzg\" (UniqueName: \"kubernetes.io/projected/408b1443-a9e2-493d-9597-1e5512dd88ff-kube-api-access-8gqzg\") pod \"authorino-c87c8b988-knzjq\" (UID: \"408b1443-a9e2-493d-9597-1e5512dd88ff\") " pod="kuadrant-system/authorino-c87c8b988-knzjq" Oct 30 19:26:10.403205 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:26:10.403170 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:26:10.479425 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.479390 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/408b1443-a9e2-493d-9597-1e5512dd88ff-tls-cert\") pod \"authorino-c87c8b988-knzjq\" (UID: \"408b1443-a9e2-493d-9597-1e5512dd88ff\") " pod="kuadrant-system/authorino-c87c8b988-knzjq" Oct 30 19:26:10.479581 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.479475 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gqzg\" (UniqueName: \"kubernetes.io/projected/408b1443-a9e2-493d-9597-1e5512dd88ff-kube-api-access-8gqzg\") pod \"authorino-c87c8b988-knzjq\" (UID: \"408b1443-a9e2-493d-9597-1e5512dd88ff\") " pod="kuadrant-system/authorino-c87c8b988-knzjq" Oct 30 19:26:10.481919 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.481899 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/408b1443-a9e2-493d-9597-1e5512dd88ff-tls-cert\") pod \"authorino-c87c8b988-knzjq\" (UID: \"408b1443-a9e2-493d-9597-1e5512dd88ff\") " pod="kuadrant-system/authorino-c87c8b988-knzjq" Oct 30 19:26:10.489859 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.489835 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gqzg\" (UniqueName: \"kubernetes.io/projected/408b1443-a9e2-493d-9597-1e5512dd88ff-kube-api-access-8gqzg\") pod \"authorino-c87c8b988-knzjq\" (UID: \"408b1443-a9e2-493d-9597-1e5512dd88ff\") " pod="kuadrant-system/authorino-c87c8b988-knzjq" Oct 30 19:26:10.586504 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.586439 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c87c8b988-knzjq" Oct 30 19:26:10.861607 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.861576 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c87c8b988-knzjq"] Oct 30 19:26:10.862884 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:26:10.862855 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod408b1443_a9e2_493d_9597_1e5512dd88ff.slice/crio-ffe5c415dd2f5acbcae66bd1b607fa6aec8258e56a35f0b734bcfbc5e54c0382 WatchSource:0}: Error finding container ffe5c415dd2f5acbcae66bd1b607fa6aec8258e56a35f0b734bcfbc5e54c0382: Status 404 returned error can't find the container with id ffe5c415dd2f5acbcae66bd1b607fa6aec8258e56a35f0b734bcfbc5e54c0382 Oct 30 19:26:10.927581 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:10.927549 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c87c8b988-knzjq" event={"ID":"408b1443-a9e2-493d-9597-1e5512dd88ff","Type":"ContainerStarted","Data":"ffe5c415dd2f5acbcae66bd1b607fa6aec8258e56a35f0b734bcfbc5e54c0382"} Oct 30 19:26:12.938130 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:12.938089 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c87c8b988-knzjq" event={"ID":"408b1443-a9e2-493d-9597-1e5512dd88ff","Type":"ContainerStarted","Data":"a4ab973917cfd0543645815211157cf29210d20c624d7cb9255d571af7bd14e0"} Oct 30 19:26:12.960298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:12.960236 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-c87c8b988-knzjq" podStartSLOduration=1.8688543640000002 podStartE2EDuration="2.960218041s" totalImagesPullingTime="1.091363677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:12.958998475 +0000 UTC m=+393.685815774" watchObservedRunningTime="2026-10-30 19:26:12.960218041 +0000 UTC m=+393.687035343" Oct 30 19:26:12.991941 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:12.991904 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c5bf6b5c9-vmtjh"] Oct 30 19:26:12.992472 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:12.992127 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" podUID="b0e9c4b8-53fa-4838-a446-94f3a5c525fc" containerName="authorino" containerID="cri-o://fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926" gracePeriod=30 Oct 30 19:26:13.251068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.251040 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:26:13.405401 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.405372 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-kube-api-access-k6vdl\" (UniqueName: \"kubernetes.io/projected/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-kube-api-access-k6vdl\") pod \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\" (UID: \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\") " Oct 30 19:26:13.405550 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.405426 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-tls-cert\" (UniqueName: \"kubernetes.io/secret/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-tls-cert\") pod \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\" (UID: \"b0e9c4b8-53fa-4838-a446-94f3a5c525fc\") " Oct 30 19:26:13.408053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.408023 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-kube-api-access-k6vdl" pod "b0e9c4b8-53fa-4838-a446-94f3a5c525fc" (UID: "b0e9c4b8-53fa-4838-a446-94f3a5c525fc"). InnerVolumeSpecName "kube-api-access-k6vdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:26:13.417927 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.417889 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-tls-cert" pod "b0e9c4b8-53fa-4838-a446-94f3a5c525fc" (UID: "b0e9c4b8-53fa-4838-a446-94f3a5c525fc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:26:13.519725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.519335 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6vdl\" (UniqueName: \"kubernetes.io/projected/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-kube-api-access-k6vdl\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:13.519725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.519376 5627 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0e9c4b8-53fa-4838-a446-94f3a5c525fc-tls-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:13.560013 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.559981 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:13.746782 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.746748 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:13.944134 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.944040 5627 generic.go:356] "Generic (PLEG): container finished" podID="b0e9c4b8-53fa-4838-a446-94f3a5c525fc" containerID="fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926" exitCode=0 Oct 30 19:26:13.944134 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.944104 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" Oct 30 19:26:13.944640 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.944092 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" event={"ID":"b0e9c4b8-53fa-4838-a446-94f3a5c525fc","Type":"ContainerDied","Data":"fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926"} Oct 30 19:26:13.944640 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.944231 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5bf6b5c9-vmtjh" event={"ID":"b0e9c4b8-53fa-4838-a446-94f3a5c525fc","Type":"ContainerDied","Data":"27be66130ddc7b4582a561ea98e74827f623631794e03ad516572d96b3fd8108"} Oct 30 19:26:13.944640 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.944258 5627 scope.go:122] "RemoveContainer" containerID="fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926" Oct 30 19:26:13.954026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.954004 5627 scope.go:122] "RemoveContainer" containerID="fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926" Oct 30 19:26:13.954303 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:26:13.954282 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926\": container with ID starting with fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926 not found: ID does not exist" containerID="fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926" Oct 30 19:26:13.954363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.954312 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926"} err="failed to get container status \"fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926\": rpc error: code = NotFound desc = could not find container \"fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926\": container with ID starting with fff2d57b3fbe30f807d092668c9849e8437144870b34ad508618445d269b1926 not found: ID does not exist" Oct 30 19:26:13.979265 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.979234 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c5bf6b5c9-vmtjh"] Oct 30 19:26:13.990465 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:13.990437 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c5bf6b5c9-vmtjh"] Oct 30 19:26:15.396988 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:15.396955 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0e9c4b8-53fa-4838-a446-94f3a5c525fc" path="/var/lib/kubelet/pods/b0e9c4b8-53fa-4838-a446-94f3a5c525fc/volumes" Oct 30 19:26:17.941444 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:17.941409 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:18.086549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:18.086516 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:21.233015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.232980 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds"] Oct 30 19:26:21.233383 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.233318 5627 cpu_manager.go:414] "containerMap: removing container" podUID="b0e9c4b8-53fa-4838-a446-94f3a5c525fc" containerName="authorino" Oct 30 19:26:21.233383 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.233329 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0e9c4b8-53fa-4838-a446-94f3a5c525fc" containerName="authorino" Oct 30 19:26:21.233464 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.233398 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0e9c4b8-53fa-4838-a446-94f3a5c525fc" containerName="authorino" Oct 30 19:26:21.239193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.239160 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.248992 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.248865 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds"] Oct 30 19:26:21.249226 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.249202 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 19:26:21.329223 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.329188 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tbg4v\"" Oct 30 19:26:21.400265 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400231 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.400413 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400268 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mjqz\" (UniqueName: \"kubernetes.io/projected/e4d6a7ba-f41e-47b5-9e16-993a271bee50-kube-api-access-7mjqz\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.400413 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400355 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.400413 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400399 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.400551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400446 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.400551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400464 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.400660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400561 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.400660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400648 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.400786 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.400686 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502066 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502029 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502070 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502101 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502168 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502194 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mjqz\" (UniqueName: \"kubernetes.io/projected/e4d6a7ba-f41e-47b5-9e16-993a271bee50-kube-api-access-7mjqz\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502231 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502503 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502271 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502503 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502346 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502503 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502371 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502771 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502869 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502847 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502866 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502898 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.502930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.502910 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.504686 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.504664 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.504686 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.504678 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.511572 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.511544 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4d6a7ba-f41e-47b5-9e16-993a271bee50-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.511727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.511684 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mjqz\" (UniqueName: \"kubernetes.io/projected/e4d6a7ba-f41e-47b5-9e16-993a271bee50-kube-api-access-7mjqz\") pod \"maas-default-gateway-openshift-default-75bf867b6c-v7rds\" (UID: \"e4d6a7ba-f41e-47b5-9e16-993a271bee50\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.557871 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.557848 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:21.832647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.832621 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds"] Oct 30 19:26:21.834404 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:26:21.834380 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d6a7ba_f41e_47b5_9e16_993a271bee50.slice/crio-2ac93d080365561b25611ce282aa65ddfd5faac03164f1eaa8f91b39e4f840de WatchSource:0}: Error finding container 2ac93d080365561b25611ce282aa65ddfd5faac03164f1eaa8f91b39e4f840de: Status 404 returned error can't find the container with id 2ac93d080365561b25611ce282aa65ddfd5faac03164f1eaa8f91b39e4f840de Oct 30 19:26:21.983968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:21.983924 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" event={"ID":"e4d6a7ba-f41e-47b5-9e16-993a271bee50","Type":"ContainerStarted","Data":"2ac93d080365561b25611ce282aa65ddfd5faac03164f1eaa8f91b39e4f840de"} Oct 30 19:26:24.699054 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:24.699026 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:24.699308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:24.699096 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:24.699308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:24.699127 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:25.000911 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:25.000873 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" event={"ID":"e4d6a7ba-f41e-47b5-9e16-993a271bee50","Type":"ContainerStarted","Data":"88440d5363b6f23053863a7c4016927448cf962fe1d56296bd11b255aa66eeb8"} Oct 30 19:26:25.029391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:25.029333 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" podStartSLOduration=1.171097434 podStartE2EDuration="4.029320899s" totalImagesPullingTime="2.858223465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:25.026981922 +0000 UTC m=+405.753799220" watchObservedRunningTime="2026-10-30 19:26:25.029320899 +0000 UTC m=+405.756138193" Oct 30 19:26:25.558516 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:25.558474 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:25.560028 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:25.560006 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-v7rds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Oct 30 19:26:25.560089 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:25.560054 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" podUID="e4d6a7ba-f41e-47b5-9e16-993a271bee50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Oct 30 19:26:26.558501 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:26.558460 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-v7rds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Oct 30 19:26:26.559047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:26.558520 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" podUID="e4d6a7ba-f41e-47b5-9e16-993a271bee50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Oct 30 19:26:27.558740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:27.558679 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-v7rds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Oct 30 19:26:27.559151 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:27.558765 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" podUID="e4d6a7ba-f41e-47b5-9e16-993a271bee50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Oct 30 19:26:28.558533 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:28.558494 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-v7rds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Oct 30 19:26:28.558730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:28.558581 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" podUID="e4d6a7ba-f41e-47b5-9e16-993a271bee50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Oct 30 19:26:29.559193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:29.559149 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-v7rds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Oct 30 19:26:29.559586 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:29.559210 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" podUID="e4d6a7ba-f41e-47b5-9e16-993a271bee50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Oct 30 19:26:30.559169 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:30.559132 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-v7rds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Oct 30 19:26:30.559348 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:30.559195 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" podUID="e4d6a7ba-f41e-47b5-9e16-993a271bee50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Oct 30 19:26:31.559177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:31.559132 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-v7rds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Oct 30 19:26:31.559353 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:31.559206 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" podUID="e4d6a7ba-f41e-47b5-9e16-993a271bee50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Oct 30 19:26:33.052363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:33.052328 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:33.053788 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:33.053755 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:33.054425 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:33.054401 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7rds" Oct 30 19:26:36.258882 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.258844 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6"] Oct 30 19:26:36.262193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.262166 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.282330 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.282307 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6"] Oct 30 19:26:36.336047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336019 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.336047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336049 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.336227 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336079 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.336227 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336192 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.336297 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336225 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzmfw\" (UniqueName: \"kubernetes.io/projected/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-kube-api-access-pzmfw\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.336329 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336314 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.336390 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336372 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.336445 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336432 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.336491 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.336452 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437367 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437332 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437367 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437368 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzmfw\" (UniqueName: \"kubernetes.io/projected/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-kube-api-access-pzmfw\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437621 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437401 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437621 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437462 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437621 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437498 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437621 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437533 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437621 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437595 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437624 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437677 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437842 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.437930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437899 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.438060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.437982 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.438106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.438050 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.438433 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.438411 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.439933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.439908 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.440193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.440176 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.447002 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.446977 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.447210 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.447189 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzmfw\" (UniqueName: \"kubernetes.io/projected/78d8bc24-bd69-4497-b7f8-cc98fcf3c379-kube-api-access-pzmfw\") pod \"maas-default-gateway-openshift-default-75bf867b6c-n5rx6\" (UID: \"78d8bc24-bd69-4497-b7f8-cc98fcf3c379\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.575048 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.574975 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:36.850445 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.850409 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6"] Oct 30 19:26:36.852666 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:26:36.852639 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d8bc24_bd69_4497_b7f8_cc98fcf3c379.slice/crio-5b114e105312d1b5684612534131b7fec316e4979d8f7719526bcd204cf1b965 WatchSource:0}: Error finding container 5b114e105312d1b5684612534131b7fec316e4979d8f7719526bcd204cf1b965: Status 404 returned error can't find the container with id 5b114e105312d1b5684612534131b7fec316e4979d8f7719526bcd204cf1b965 Oct 30 19:26:36.856215 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.856184 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:36.856315 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.856259 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:36.856315 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:36.856296 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:37.078877 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:37.078840 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" event={"ID":"78d8bc24-bd69-4497-b7f8-cc98fcf3c379","Type":"ContainerStarted","Data":"f55ed34c39776a604ca54c699483ef623480f60c784624fd55132edc4beafb05"} Oct 30 19:26:37.079067 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:37.078924 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" event={"ID":"78d8bc24-bd69-4497-b7f8-cc98fcf3c379","Type":"ContainerStarted","Data":"5b114e105312d1b5684612534131b7fec316e4979d8f7719526bcd204cf1b965"} Oct 30 19:26:37.100580 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:37.100477 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podStartSLOduration=1.100459182 podStartE2EDuration="1.100459182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:37.099185543 +0000 UTC m=+417.826002843" watchObservedRunningTime="2026-10-30 19:26:37.100459182 +0000 UTC m=+417.827276480" Oct 30 19:26:37.575319 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:37.575281 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:37.576873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:37.576848 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Oct 30 19:26:37.576999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:37.576908 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Oct 30 19:26:38.576205 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:38.576173 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Oct 30 19:26:38.576594 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:38.576225 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Oct 30 19:26:39.575380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:39.575342 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Oct 30 19:26:39.575551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:39.575402 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Oct 30 19:26:40.402311 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:26:40.402274 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:26:40.575435 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:40.575401 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Oct 30 19:26:40.575607 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:40.575455 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Oct 30 19:26:41.575956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:41.575924 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Oct 30 19:26:41.576326 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:41.575975 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Oct 30 19:26:42.575829 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:42.575789 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Oct 30 19:26:42.576007 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:42.575848 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Oct 30 19:26:43.273120 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:43.273072 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:43.576189 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:43.576103 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Oct 30 19:26:43.576189 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:43.576172 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Oct 30 19:26:43.774451 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:43.774420 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:44.700688 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:44.700648 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:45.400643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:45.400609 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:45.401002 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:45.400978 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:45.401889 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:45.401869 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" Oct 30 19:26:45.487405 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:45.487367 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:45.633015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:45.632980 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:47.033127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:47.033088 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:47.173099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:47.173056 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:51.273085 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.273053 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh"] Oct 30 19:26:51.278308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.278287 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.290412 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.290386 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh"] Oct 30 19:26:51.376201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376160 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.376201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376200 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.376429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376347 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vl8s\" (UniqueName: \"kubernetes.io/projected/177c6297-5b36-4a86-9baf-21793a0ec95d-kube-api-access-8vl8s\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.376429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376381 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.376429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376403 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.376429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376424 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/177c6297-5b36-4a86-9baf-21793a0ec95d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.376649 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376547 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.376649 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376612 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.376795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.376675 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.477694 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.477661 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vl8s\" (UniqueName: \"kubernetes.io/projected/177c6297-5b36-4a86-9baf-21793a0ec95d-kube-api-access-8vl8s\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.477694 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.477695 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.477957 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.477735 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.477957 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.477850 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/177c6297-5b36-4a86-9baf-21793a0ec95d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.477957 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.477902 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478128 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.477961 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478128 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.478027 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478128 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.478059 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478128 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.478076 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478128 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.478091 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478388 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.478184 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478388 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.478374 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478501 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.478405 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.478612 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.478586 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/177c6297-5b36-4a86-9baf-21793a0ec95d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.480330 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.480303 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.480573 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.480556 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.487030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.487005 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/177c6297-5b36-4a86-9baf-21793a0ec95d-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.487330 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.487307 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vl8s\" (UniqueName: \"kubernetes.io/projected/177c6297-5b36-4a86-9baf-21793a0ec95d-kube-api-access-8vl8s\") pod \"maas-default-gateway-openshift-default-75bf867b6c-dlrbh\" (UID: \"177c6297-5b36-4a86-9baf-21793a0ec95d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.591659 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.591586 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:51.866229 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.866197 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh"] Oct 30 19:26:51.867435 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:26:51.867405 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177c6297_5b36_4a86_9baf_21793a0ec95d.slice/crio-f19f2df8e6358e83b8e1b85567e5fc23bf42ddcc6f1660a7e1ddf1d1b2544baf WatchSource:0}: Error finding container f19f2df8e6358e83b8e1b85567e5fc23bf42ddcc6f1660a7e1ddf1d1b2544baf: Status 404 returned error can't find the container with id f19f2df8e6358e83b8e1b85567e5fc23bf42ddcc6f1660a7e1ddf1d1b2544baf Oct 30 19:26:51.869934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.869903 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:51.870019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.869971 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:51.870019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:51.870003 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:26:52.152296 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:52.152203 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" event={"ID":"177c6297-5b36-4a86-9baf-21793a0ec95d","Type":"ContainerStarted","Data":"c7aa34630c8add7d6ed159a8765bd27995a26451ccc45be2794fe9ae835b3674"} Oct 30 19:26:52.152296 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:52.152264 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" event={"ID":"177c6297-5b36-4a86-9baf-21793a0ec95d","Type":"ContainerStarted","Data":"f19f2df8e6358e83b8e1b85567e5fc23bf42ddcc6f1660a7e1ddf1d1b2544baf"} Oct 30 19:26:52.177602 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:52.177545 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podStartSLOduration=1.177533109 podStartE2EDuration="1.177533109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:52.174980913 +0000 UTC m=+432.901798213" watchObservedRunningTime="2026-10-30 19:26:52.177533109 +0000 UTC m=+432.904350479" Oct 30 19:26:52.591910 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:52.591875 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:26:52.592996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:52.592974 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Oct 30 19:26:52.593054 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:52.593033 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Oct 30 19:26:53.592921 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:53.592884 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Oct 30 19:26:53.593298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:53.592940 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Oct 30 19:26:54.592259 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:54.592215 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Oct 30 19:26:54.592450 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:54.592284 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Oct 30 19:26:55.592389 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:55.592350 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Oct 30 19:26:55.592852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:55.592406 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Oct 30 19:26:56.592302 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:56.592267 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Oct 30 19:26:56.592477 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:56.592321 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Oct 30 19:26:57.591999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:57.591960 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Oct 30 19:26:57.592208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:57.592030 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Oct 30 19:26:58.592134 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:58.592100 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Oct 30 19:26:58.592518 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:26:58.592155 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Oct 30 19:27:00.338805 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:00.338766 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:27:00.339259 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:00.339169 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:27:00.340055 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:00.340031 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" Oct 30 19:27:01.144534 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:01.144503 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:10.402146 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:27:10.402100 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:27:27.128600 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.128525 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:27.279553 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.279517 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:27.660587 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.660551 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:27.669612 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.669576 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc"] Oct 30 19:27:27.673452 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.673431 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.686512 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.686496 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-f7f075-openshift-default-dockercfg-p74l2\"" Oct 30 19:27:27.698758 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.698735 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc"] Oct 30 19:27:27.716962 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.716940 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.717061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.716970 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nckfs\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-kube-api-access-nckfs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.717061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.717011 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77a4921d-b562-4871-b94c-ae5896d7a997-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.717181 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.717099 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.717181 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.717168 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.717255 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.717215 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77a4921d-b562-4871-b94c-ae5896d7a997-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.717318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.717300 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.717363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.717348 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.717407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.717385 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.768300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.768272 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m"] Oct 30 19:27:27.771895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.771876 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.787596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.787576 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-639c15-openshift-default-dockercfg-vrz5l\"" Oct 30 19:27:27.795614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.795595 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m"] Oct 30 19:27:27.818133 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818110 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.818293 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818143 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.818293 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818161 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.818293 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818190 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.818293 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818261 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nckfs\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-kube-api-access-nckfs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.818766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818382 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77a4921d-b562-4871-b94c-ae5896d7a997-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.818766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818419 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.818766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818476 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.818766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818502 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.818766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818538 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.818766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818567 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67pn9\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-kube-api-access-67pn9\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.818766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818594 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.818766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818621 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77a4921d-b562-4871-b94c-ae5896d7a997-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.819201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.818981 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.819201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.819030 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.819201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.819117 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.819201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.819136 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.819201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.819159 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.819201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.819185 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.819518 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.819210 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77a4921d-b562-4871-b94c-ae5896d7a997-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.819802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.819782 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.821030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.821012 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.821270 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.821230 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.821923 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.821901 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:27.822031 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.821984 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.822110 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.822090 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77a4921d-b562-4871-b94c-ae5896d7a997-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.827612 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.827591 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nckfs\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-kube-api-access-nckfs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.829900 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.829878 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:27.920168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920095 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920132 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920166 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920202 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920224 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920249 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920330 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920357 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920381 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67pn9\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-kube-api-access-67pn9\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920680 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920608 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920742 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920757 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.920859 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.920827 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.921175 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.921092 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.922835 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.922816 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.922925 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.922821 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.931888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.931867 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.934438 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.934416 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67pn9\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-kube-api-access-67pn9\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:27.986028 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:27.986003 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:28.030239 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.030202 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:28.083969 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.083943 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:28.223443 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.223367 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:28.269183 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.269159 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc"] Oct 30 19:27:28.271390 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:28.271367 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a4921d_b562_4871_b94c_ae5896d7a997.slice/crio-bf9bdbc5d608144bb9cc74e741f8ffdfa9469d9ff531d39b9efde3462eb859ee WatchSource:0}: Error finding container bf9bdbc5d608144bb9cc74e741f8ffdfa9469d9ff531d39b9efde3462eb859ee: Status 404 returned error can't find the container with id bf9bdbc5d608144bb9cc74e741f8ffdfa9469d9ff531d39b9efde3462eb859ee Oct 30 19:27:28.273427 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.273401 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:28.273492 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.273469 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:28.273525 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.273508 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:28.315509 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.315476 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" event={"ID":"77a4921d-b562-4871-b94c-ae5896d7a997","Type":"ContainerStarted","Data":"bf9bdbc5d608144bb9cc74e741f8ffdfa9469d9ff531d39b9efde3462eb859ee"} Oct 30 19:27:28.376916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.376891 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m"] Oct 30 19:27:28.380074 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:28.379861 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e2a9bf_d40f_48b8_9fd1_0b3e67ca7317.slice/crio-72eb99f262a41427baef1f5336b2f4ac1801bf5f2fb72393fe599a9b87aebe27 WatchSource:0}: Error finding container 72eb99f262a41427baef1f5336b2f4ac1801bf5f2fb72393fe599a9b87aebe27: Status 404 returned error can't find the container with id 72eb99f262a41427baef1f5336b2f4ac1801bf5f2fb72393fe599a9b87aebe27 Oct 30 19:27:28.382555 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.382528 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:28.382675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.382591 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:28.382675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.382619 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:28.384089 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.384063 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:28.609692 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:28.609494 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:29.321691 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:29.321646 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" event={"ID":"77a4921d-b562-4871-b94c-ae5896d7a997","Type":"ContainerStarted","Data":"22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7"} Oct 30 19:27:29.323243 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:29.323216 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" event={"ID":"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317","Type":"ContainerStarted","Data":"a0a9770176072a6a9164006fb1330271e8f90bb901f43cbe621d8803b2950651"} Oct 30 19:27:29.323351 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:29.323262 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" event={"ID":"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317","Type":"ContainerStarted","Data":"72eb99f262a41427baef1f5336b2f4ac1801bf5f2fb72393fe599a9b87aebe27"} Oct 30 19:27:29.345320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:29.345262 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" podStartSLOduration=2.345245859 podStartE2EDuration="2.345245859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:29.344624682 +0000 UTC m=+470.071441972" watchObservedRunningTime="2026-10-30 19:27:29.345245859 +0000 UTC m=+470.072063157" Oct 30 19:27:29.366830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:29.366791 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" podStartSLOduration=2.366778737 podStartE2EDuration="2.366778737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:29.365801065 +0000 UTC m=+470.092618363" watchObservedRunningTime="2026-10-30 19:27:29.366778737 +0000 UTC m=+470.093596111" Oct 30 19:27:29.986786 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:29.986753 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:29.991985 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:29.991954 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:30.084221 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:30.084184 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:30.089493 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:30.089467 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:30.328564 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:30.328532 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:30.329027 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:30.328589 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:30.329483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:30.329466 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:27:30.329589 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:30.329514 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:27:31.574461 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:31.574427 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:31.746400 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:31.746337 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:31.911636 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:31.911554 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:32.045534 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.045498 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:32.191614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.191532 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:32.469126 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.469047 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:32.595165 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.595138 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:32.795816 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.795777 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:32.870170 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.870139 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk"] Oct 30 19:27:32.873434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.873410 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:32.888656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.888631 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk"] Oct 30 19:27:32.927852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.927827 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-3bc085\"" Oct 30 19:27:32.927970 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.927865 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-3bc085-dockercfg-b6l4v\"" Oct 30 19:27:32.964479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.964439 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:32.973108 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.973040 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3f421b-8c94-4018-92c1-f5655816d945-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk\" (UID: \"8e3f421b-8c94-4018-92c1-f5655816d945\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:32.973678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:32.973653 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4zh5\" (UniqueName: \"kubernetes.io/projected/8e3f421b-8c94-4018-92c1-f5655816d945-kube-api-access-g4zh5\") pod \"payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk\" (UID: \"8e3f421b-8c94-4018-92c1-f5655816d945\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:33.075187 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.075112 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4zh5\" (UniqueName: \"kubernetes.io/projected/8e3f421b-8c94-4018-92c1-f5655816d945-kube-api-access-g4zh5\") pod \"payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk\" (UID: \"8e3f421b-8c94-4018-92c1-f5655816d945\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:33.075187 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.075160 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3f421b-8c94-4018-92c1-f5655816d945-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk\" (UID: \"8e3f421b-8c94-4018-92c1-f5655816d945\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:33.075806 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.075789 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3f421b-8c94-4018-92c1-f5655816d945-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk\" (UID: \"8e3f421b-8c94-4018-92c1-f5655816d945\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:33.088691 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.088667 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4zh5\" (UniqueName: \"kubernetes.io/projected/8e3f421b-8c94-4018-92c1-f5655816d945-kube-api-access-g4zh5\") pod \"payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk\" (UID: \"8e3f421b-8c94-4018-92c1-f5655816d945\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:33.188000 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.187975 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:33.475467 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.475443 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk"] Oct 30 19:27:33.476964 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:33.476929 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e3f421b_8c94_4018_92c1_f5655816d945.slice/crio-9ba9886c8fbc6c1a841aa335cd791db091a52268999b49ef1b44e8a52025ad9c WatchSource:0}: Error finding container 9ba9886c8fbc6c1a841aa335cd791db091a52268999b49ef1b44e8a52025ad9c: Status 404 returned error can't find the container with id 9ba9886c8fbc6c1a841aa335cd791db091a52268999b49ef1b44e8a52025ad9c Oct 30 19:27:33.814101 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.814067 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:33.890945 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.890838 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv"] Oct 30 19:27:33.895570 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.895539 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:33.907956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.907932 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv"] Oct 30 19:27:33.979960 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.979937 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-6cf459-dockercfg-g4hx4\"" Oct 30 19:27:33.985317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.985293 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bhfq\" (UniqueName: \"kubernetes.io/projected/1668db65-90fe-4e2f-9ae9-70c50a2c788a-kube-api-access-5bhfq\") pod \"payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv\" (UID: \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:33.985412 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.985327 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1668db65-90fe-4e2f-9ae9-70c50a2c788a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv\" (UID: \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:33.988049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:33.988027 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-6cf459\"" Oct 30 19:27:34.022117 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.022091 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:34.086884 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.086791 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bhfq\" (UniqueName: \"kubernetes.io/projected/1668db65-90fe-4e2f-9ae9-70c50a2c788a-kube-api-access-5bhfq\") pod \"payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv\" (UID: \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:34.086884 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.086834 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1668db65-90fe-4e2f-9ae9-70c50a2c788a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv\" (UID: \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:34.087845 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.087824 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1668db65-90fe-4e2f-9ae9-70c50a2c788a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv\" (UID: \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:34.100548 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.100519 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bhfq\" (UniqueName: \"kubernetes.io/projected/1668db65-90fe-4e2f-9ae9-70c50a2c788a-kube-api-access-5bhfq\") pod \"payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv\" (UID: \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:34.211149 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.211117 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:34.353920 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.353837 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" event={"ID":"8e3f421b-8c94-4018-92c1-f5655816d945","Type":"ContainerStarted","Data":"9ba9886c8fbc6c1a841aa335cd791db091a52268999b49ef1b44e8a52025ad9c"} Oct 30 19:27:34.368672 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.368645 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2"] Oct 30 19:27:34.372077 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.372061 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.379279 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.379256 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-3bc085-serving-cert\"" Oct 30 19:27:34.387240 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.387217 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2"] Oct 30 19:27:34.478668 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.478626 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:34.490804 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.490773 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv8ts\" (UniqueName: \"kubernetes.io/projected/e75aa317-1733-4e9f-af92-163add3ff6ab-kube-api-access-mv8ts\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.490950 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.490822 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-metrics-certs\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.490950 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.490916 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75aa317-1733-4e9f-af92-163add3ff6ab-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.491046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.490971 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.505159 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.505109 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv"] Oct 30 19:27:34.507503 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:34.507476 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1668db65_90fe_4e2f_9ae9_70c50a2c788a.slice/crio-c256c204f7431b598c778cce343203131682f9656b1cdbc8ce950c80322dfde1 WatchSource:0}: Error finding container c256c204f7431b598c778cce343203131682f9656b1cdbc8ce950c80322dfde1: Status 404 returned error can't find the container with id c256c204f7431b598c778cce343203131682f9656b1cdbc8ce950c80322dfde1 Oct 30 19:27:34.592433 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.592397 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv8ts\" (UniqueName: \"kubernetes.io/projected/e75aa317-1733-4e9f-af92-163add3ff6ab-kube-api-access-mv8ts\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.592433 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.592438 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-metrics-certs\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.592690 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.592574 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75aa317-1733-4e9f-af92-163add3ff6ab-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.592690 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.592620 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.593266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.593240 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75aa317-1733-4e9f-af92-163add3ff6ab-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.595613 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.595587 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.595776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.595602 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-metrics-certs\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.605082 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.605024 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv8ts\" (UniqueName: \"kubernetes.io/projected/e75aa317-1733-4e9f-af92-163add3ff6ab-kube-api-access-mv8ts\") pod \"maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.653374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.653343 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:34.688324 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.688303 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:34.985949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:34.985923 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2"] Oct 30 19:27:34.987406 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:34.987373 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75aa317_1733_4e9f_af92_163add3ff6ab.slice/crio-16e87a221beac7d4825ae2b4ed33eb46dab7b138c033b05b1b0c190666ccedf2 WatchSource:0}: Error finding container 16e87a221beac7d4825ae2b4ed33eb46dab7b138c033b05b1b0c190666ccedf2: Status 404 returned error can't find the container with id 16e87a221beac7d4825ae2b4ed33eb46dab7b138c033b05b1b0c190666ccedf2 Oct 30 19:27:35.077084 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:35.077051 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:35.250950 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:35.250906 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:35.362798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:35.362680 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" event={"ID":"1668db65-90fe-4e2f-9ae9-70c50a2c788a","Type":"ContainerStarted","Data":"c256c204f7431b598c778cce343203131682f9656b1cdbc8ce950c80322dfde1"} Oct 30 19:27:35.364386 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:35.364345 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" event={"ID":"e75aa317-1733-4e9f-af92-163add3ff6ab","Type":"ContainerStarted","Data":"16e87a221beac7d4825ae2b4ed33eb46dab7b138c033b05b1b0c190666ccedf2"} Oct 30 19:27:35.494176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:35.494104 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:35.863532 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:35.863495 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:36.114377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.114310 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:36.273191 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.273149 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:36.370629 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.370547 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" event={"ID":"8e3f421b-8c94-4018-92c1-f5655816d945","Type":"ContainerStarted","Data":"2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862"} Oct 30 19:27:36.370812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.370656 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:36.372248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.372227 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" event={"ID":"e75aa317-1733-4e9f-af92-163add3ff6ab","Type":"ContainerStarted","Data":"82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1"} Oct 30 19:27:36.372392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.372326 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:36.373677 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.373660 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" event={"ID":"1668db65-90fe-4e2f-9ae9-70c50a2c788a","Type":"ContainerStarted","Data":"fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43"} Oct 30 19:27:36.373780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.373761 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:36.390860 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.390788 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" podStartSLOduration=1.765955367 podStartE2EDuration="4.390774087s" totalImagesPullingTime="2.62481872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:36.389795106 +0000 UTC m=+477.116612404" watchObservedRunningTime="2026-10-30 19:27:36.390774087 +0000 UTC m=+477.117591387" Oct 30 19:27:36.412768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.412732 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" podStartSLOduration=1.760251635 podStartE2EDuration="3.412694462s" totalImagesPullingTime="1.652442827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:36.411394059 +0000 UTC m=+477.138211358" watchObservedRunningTime="2026-10-30 19:27:36.412694462 +0000 UTC m=+477.139511771" Oct 30 19:27:36.435026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.434983 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" podStartSLOduration=2.013689196 podStartE2EDuration="2.434972689s" totalImagesPullingTime="421.283493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:36.433753984 +0000 UTC m=+477.160571283" watchObservedRunningTime="2026-10-30 19:27:36.434972689 +0000 UTC m=+477.161789986" Oct 30 19:27:36.837863 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.837831 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws"] Oct 30 19:27:36.841352 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.841336 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:36.853834 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.853809 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-6cf459-serving-cert\"" Oct 30 19:27:36.856345 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.856321 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws"] Oct 30 19:27:36.915652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.915628 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d59d58-fa19-4884-9d5a-5583b2758491-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:36.915780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.915668 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9jp9\" (UniqueName: \"kubernetes.io/projected/e8d59d58-fa19-4884-9d5a-5583b2758491-kube-api-access-p9jp9\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:36.915780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.915770 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-metrics-certs\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:36.915881 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.915867 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:36.934724 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:36.934678 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:37.017079 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.017038 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.017265 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.017138 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d59d58-fa19-4884-9d5a-5583b2758491-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.017265 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.017173 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9jp9\" (UniqueName: \"kubernetes.io/projected/e8d59d58-fa19-4884-9d5a-5583b2758491-kube-api-access-p9jp9\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.017265 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:27:37.017194 5627 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-serving-cert: secret "maas-api-e2e-worker-w1-6cf459-serving-cert" not found Oct 30 19:27:37.017265 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.017217 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-metrics-certs\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.017511 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:27:37.017289 5627 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls podName:e8d59d58-fa19-4884-9d5a-5583b2758491 nodeName:}" failed. No retries permitted until 2026-10-30 19:27:37.517268131 +0000 UTC m=+478.244085421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls") pod "maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" (UID: "e8d59d58-fa19-4884-9d5a-5583b2758491") : secret "maas-api-e2e-worker-w1-6cf459-serving-cert" not found Oct 30 19:27:37.017888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.017865 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d59d58-fa19-4884-9d5a-5583b2758491-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.019802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.019782 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-metrics-certs\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.028770 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.028751 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9jp9\" (UniqueName: \"kubernetes.io/projected/e8d59d58-fa19-4884-9d5a-5583b2758491-kube-api-access-p9jp9\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.084936 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.084913 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:37.221150 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.221052 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:37.377473 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.377442 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:37.523410 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.523373 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.526023 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.526000 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls\") pod \"maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.757737 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.757683 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:37.837371 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:37.837329 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:38.184208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.184177 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws"] Oct 30 19:27:38.184454 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:38.184427 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d59d58_fa19_4884_9d5a_5583b2758491.slice/crio-a886782596804cc793e877246484765d4b26857322509222ee421d918853232b WatchSource:0}: Error finding container a886782596804cc793e877246484765d4b26857322509222ee421d918853232b: Status 404 returned error can't find the container with id a886782596804cc793e877246484765d4b26857322509222ee421d918853232b Oct 30 19:27:38.215745 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.215714 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp"] Oct 30 19:27:38.220438 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.220422 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.227372 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.227347 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-171808-serving-cert\"" Oct 30 19:27:38.230178 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.230157 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:38.236864 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.236839 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp"] Oct 30 19:27:38.334790 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.334749 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-metrics-certs\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.334999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.334818 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msrt5\" (UniqueName: \"kubernetes.io/projected/512cbb25-1c9d-4419-a15c-d954feb58d31-kube-api-access-msrt5\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.334999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.334944 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512cbb25-1c9d-4419-a15c-d954feb58d31-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.334999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.334980 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-maas-api-tls\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.385996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.385962 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" event={"ID":"e8d59d58-fa19-4884-9d5a-5583b2758491","Type":"ContainerStarted","Data":"a886782596804cc793e877246484765d4b26857322509222ee421d918853232b"} Oct 30 19:27:38.436431 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.436338 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-metrics-certs\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.436431 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.436390 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msrt5\" (UniqueName: \"kubernetes.io/projected/512cbb25-1c9d-4419-a15c-d954feb58d31-kube-api-access-msrt5\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.436669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.436486 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512cbb25-1c9d-4419-a15c-d954feb58d31-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.436669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.436514 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-maas-api-tls\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.437457 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.437430 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512cbb25-1c9d-4419-a15c-d954feb58d31-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.438996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.438975 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-metrics-certs\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.439093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.439075 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-maas-api-tls\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.452149 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.452128 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msrt5\" (UniqueName: \"kubernetes.io/projected/512cbb25-1c9d-4419-a15c-d954feb58d31-kube-api-access-msrt5\") pod \"maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.456297 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.456266 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:38.535918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.535884 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:38.816364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:38.816033 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp"] Oct 30 19:27:38.819907 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:38.819877 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod512cbb25_1c9d_4419_a15c_d954feb58d31.slice/crio-be7bca5a7b9af22ab5ba19df1e03c5c3b1ba8c9ba07049b8e3f1845579f861bc WatchSource:0}: Error finding container be7bca5a7b9af22ab5ba19df1e03c5c3b1ba8c9ba07049b8e3f1845579f861bc: Status 404 returned error can't find the container with id be7bca5a7b9af22ab5ba19df1e03c5c3b1ba8c9ba07049b8e3f1845579f861bc Oct 30 19:27:39.257395 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.257362 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm"] Oct 30 19:27:39.261729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.261681 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.266931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.266904 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-d55178-serving-cert\"" Oct 30 19:27:39.270219 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.270186 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm"] Oct 30 19:27:39.340510 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.340485 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:39.352174 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.352139 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7b86-e8ee-4b9f-a695-92080e70f540-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.352322 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.352214 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrbrt\" (UniqueName: \"kubernetes.io/projected/d18b7b86-e8ee-4b9f-a695-92080e70f540-kube-api-access-wrbrt\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.352322 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.352289 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-maas-api-tls\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.352435 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.352421 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-metrics-certs\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.398315 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.398284 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" event={"ID":"e8d59d58-fa19-4884-9d5a-5583b2758491","Type":"ContainerStarted","Data":"28c6e970cf241bc91071e23996c86626421d5eeced8f311c6444bc5ab2ccda83"} Oct 30 19:27:39.398466 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.398345 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:39.398466 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.398361 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" event={"ID":"512cbb25-1c9d-4419-a15c-d954feb58d31","Type":"ContainerStarted","Data":"2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241"} Oct 30 19:27:39.398466 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.398378 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:39.398466 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.398391 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" event={"ID":"512cbb25-1c9d-4419-a15c-d954feb58d31","Type":"ContainerStarted","Data":"be7bca5a7b9af22ab5ba19df1e03c5c3b1ba8c9ba07049b8e3f1845579f861bc"} Oct 30 19:27:39.419202 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.419149 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" podStartSLOduration=2.946704477 podStartE2EDuration="3.419135556s" totalImagesPullingTime="472.431079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:39.417918843 +0000 UTC m=+480.144736141" watchObservedRunningTime="2026-10-30 19:27:39.419135556 +0000 UTC m=+480.145952852" Oct 30 19:27:39.442951 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.442896 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" podStartSLOduration=0.968155377 podStartE2EDuration="1.442878889s" totalImagesPullingTime="474.723512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:39.439813761 +0000 UTC m=+480.166631057" watchObservedRunningTime="2026-10-30 19:27:39.442878889 +0000 UTC m=+480.169696187" Oct 30 19:27:39.453406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.453363 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-maas-api-tls\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.453574 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.453448 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-metrics-certs\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.453574 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.453482 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7b86-e8ee-4b9f-a695-92080e70f540-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.453574 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.453509 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrbrt\" (UniqueName: \"kubernetes.io/projected/d18b7b86-e8ee-4b9f-a695-92080e70f540-kube-api-access-wrbrt\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.454197 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.454172 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7b86-e8ee-4b9f-a695-92080e70f540-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.456453 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.456420 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-metrics-certs\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.456553 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.456530 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-d55178-serving-cert\"" Oct 30 19:27:39.463494 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.463475 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrbrt\" (UniqueName: \"kubernetes.io/projected/d18b7b86-e8ee-4b9f-a695-92080e70f540-kube-api-access-wrbrt\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.465057 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.465039 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:39.466122 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.466107 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-maas-api-tls\") pod \"maas-api-e2e-worker-w2-d55178-666654c97-g82tm\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.586278 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.586191 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:39.872631 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:39.872555 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm"] Oct 30 19:27:39.877284 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:39.877255 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18b7b86_e8ee_4b9f_a695_92080e70f540.slice/crio-1a380b371383f0e40a25275053951a841673c40df20f2d33d544ff918153a855 WatchSource:0}: Error finding container 1a380b371383f0e40a25275053951a841673c40df20f2d33d544ff918153a855: Status 404 returned error can't find the container with id 1a380b371383f0e40a25275053951a841673c40df20f2d33d544ff918153a855 Oct 30 19:27:40.327905 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:40.327874 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:40.402130 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:40.402084 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" event={"ID":"d18b7b86-e8ee-4b9f-a695-92080e70f540","Type":"ContainerStarted","Data":"85786bcffe0acddac0d7e30b82990b5de2a4031758365f91fa698d9cce507a03"} Oct 30 19:27:40.402560 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:40.402541 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:40.402676 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:40.402589 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" event={"ID":"d18b7b86-e8ee-4b9f-a695-92080e70f540","Type":"ContainerStarted","Data":"1a380b371383f0e40a25275053951a841673c40df20f2d33d544ff918153a855"} Oct 30 19:27:40.407448 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:27:40.407420 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:27:40.427376 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:40.427334 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" podStartSLOduration=1.032503185 podStartE2EDuration="1.427321501s" totalImagesPullingTime="394.818316ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:40.426031412 +0000 UTC m=+481.152848709" watchObservedRunningTime="2026-10-30 19:27:40.427321501 +0000 UTC m=+481.154138797" Oct 30 19:27:41.161608 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:41.161564 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:41.314936 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:41.314908 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:41.932491 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:41.932462 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt"] Oct 30 19:27:41.936502 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:41.936484 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:41.948426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:41.948396 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-f7f075-serving-cert\"" Oct 30 19:27:41.950914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:41.950884 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt"] Oct 30 19:27:42.023692 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.023655 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:42.079865 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.079834 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-maas-api-tls\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.080047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.079909 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a3c20c-9338-4bce-a358-33a1d752ff46-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.080047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.080000 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7jhz\" (UniqueName: \"kubernetes.io/projected/71a3c20c-9338-4bce-a358-33a1d752ff46-kube-api-access-k7jhz\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.080047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.080025 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-metrics-certs\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.166953 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.166920 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:42.181099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.181075 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-maas-api-tls\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.181228 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.181130 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a3c20c-9338-4bce-a358-33a1d752ff46-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.181319 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.181297 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7jhz\" (UniqueName: \"kubernetes.io/projected/71a3c20c-9338-4bce-a358-33a1d752ff46-kube-api-access-k7jhz\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.181377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.181339 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-metrics-certs\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.181829 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.181808 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a3c20c-9338-4bce-a358-33a1d752ff46-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.183838 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.183792 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-maas-api-tls\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.183838 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.183828 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-metrics-certs\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.198243 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.198218 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7jhz\" (UniqueName: \"kubernetes.io/projected/71a3c20c-9338-4bce-a358-33a1d752ff46-kube-api-access-k7jhz\") pod \"maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.252224 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.252191 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:42.386290 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.386265 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:27:42.550903 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:42.550878 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt"] Oct 30 19:27:42.552559 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:27:42.552530 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a3c20c_9338_4bce_a358_33a1d752ff46.slice/crio-b322597f5d22cc0854e1fa6a40049ee736c357c60236f99e555ffe85953b6dc0 WatchSource:0}: Error finding container b322597f5d22cc0854e1fa6a40049ee736c357c60236f99e555ffe85953b6dc0: Status 404 returned error can't find the container with id b322597f5d22cc0854e1fa6a40049ee736c357c60236f99e555ffe85953b6dc0 Oct 30 19:27:43.364907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:43.364859 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:43.416991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:43.416957 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" event={"ID":"71a3c20c-9338-4bce-a358-33a1d752ff46","Type":"ContainerStarted","Data":"5f3d3abe4893cac10b282d9f871c1a6a19317cd1fea0c7e70874f9bf7bd14fc8"} Oct 30 19:27:43.416991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:43.417006 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" event={"ID":"71a3c20c-9338-4bce-a358-33a1d752ff46","Type":"ContainerStarted","Data":"b322597f5d22cc0854e1fa6a40049ee736c357c60236f99e555ffe85953b6dc0"} Oct 30 19:27:43.417247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:43.417069 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:43.480212 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:43.480162 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" podStartSLOduration=2.093522299 podStartE2EDuration="2.480149174s" totalImagesPullingTime="386.626875ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:43.478361611 +0000 UTC m=+484.205178909" watchObservedRunningTime="2026-10-30 19:27:43.480149174 +0000 UTC m=+484.206966471" Oct 30 19:27:43.540940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:43.540905 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:44.430583 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:44.430554 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:44.809248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:44.809218 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:45.407451 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:45.407422 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:27:45.407934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:45.407914 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:27:45.418020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:45.417993 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:46.412314 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:46.412283 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:27:46.813903 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:46.813860 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:47.383588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:47.383558 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:27:47.383767 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:47.383623 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:27:48.438599 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:48.438558 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:49.464961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:49.464921 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:27:49.568161 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:49.568129 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:50.760728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:50.760202 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:51.502993 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:51.502951 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:51.634728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:51.634139 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:51.773420 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:51.773332 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:52.171303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:52.171217 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:52.344152 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:52.344114 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:52.496142 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:52.496104 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:53.503796 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:53.503766 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:54.211262 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:54.211227 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:54.724214 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:54.724164 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:55.043076 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:55.043041 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:56.159553 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:56.159505 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:57.132270 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:57.132234 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:57.251273 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:57.251240 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:57.441033 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:57.440960 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:57.675222 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:57.675190 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:57.825552 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:57.825525 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:57.969302 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:57.969262 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:58.302814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:58.302779 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:58.510748 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:58.510541 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:58.530974 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:58.530939 5627 generic.go:356] "Generic (PLEG): container finished" podID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerID="a0a9770176072a6a9164006fb1330271e8f90bb901f43cbe621d8803b2950651" exitCode=0 Oct 30 19:27:58.531154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:58.531010 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" event={"ID":"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317","Type":"ContainerDied","Data":"a0a9770176072a6a9164006fb1330271e8f90bb901f43cbe621d8803b2950651"} Oct 30 19:27:58.531469 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:58.531452 5627 scope.go:122] "RemoveContainer" containerID="a0a9770176072a6a9164006fb1330271e8f90bb901f43cbe621d8803b2950651" Oct 30 19:27:58.533059 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:58.533030 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:58.533163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:58.533117 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:58.533233 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:58.533167 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:59.249788 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:59.249746 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:59.548451 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:59.548368 5627 generic.go:356] "Generic (PLEG): container finished" podID="77a4921d-b562-4871-b94c-ae5896d7a997" containerID="22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7" exitCode=0 Oct 30 19:27:59.548900 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:59.548458 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" event={"ID":"77a4921d-b562-4871-b94c-ae5896d7a997","Type":"ContainerDied","Data":"22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7"} Oct 30 19:27:59.548965 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:59.548931 5627 scope.go:122] "RemoveContainer" containerID="22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7" Oct 30 19:27:59.551718 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:59.551311 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:59.551718 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:59.551630 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:59.551822 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:59.551696 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:27:59.552273 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:27:59.552255 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" event={"ID":"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317","Type":"ContainerStarted","Data":"9cdda0f05c77105a42ac39a01eb692a892ebd4e0d62eb9f7d07ae887f292f727"} Oct 30 19:28:00.084423 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.084392 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:28:00.085448 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.085423 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:28:00.085601 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.085477 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:28:00.331847 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.331806 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:28:00.336806 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.336731 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:28:00.414248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.414195 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:00.574307 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.574263 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" event={"ID":"77a4921d-b562-4871-b94c-ae5896d7a997","Type":"ContainerStarted","Data":"4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4"} Oct 30 19:28:00.799572 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.799494 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:00.987060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.987027 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:28:00.988026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.988002 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:28:00.988149 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:00.988060 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:28:01.084519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.084439 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:28:01.084519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.084501 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:28:01.450252 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.450121 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:01.886126 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.885688 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln"] Oct 30 19:28:01.890657 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.890636 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:01.907307 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.907259 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"kube-root-ca.crt\"" Oct 30 19:28:01.907858 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.907828 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"default-dockercfg-jsz2z\"" Oct 30 19:28:01.908132 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.908115 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"openshift-service-ca.crt\"" Oct 30 19:28:01.908298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.908266 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"simulator-w1-6cf459-kserve-self-signed-certs\"" Oct 30 19:28:01.909531 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.908986 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln"] Oct 30 19:28:01.970984 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.970943 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:01.987155 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.987125 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:28:01.987327 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.987181 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:28:01.991694 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.991671 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-dshm\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:01.991825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.991798 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-home\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:01.991825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.991819 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-model-cache\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:01.991909 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.991889 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tls-certs\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:01.991948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.991923 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tmp-dir\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:01.992043 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:01.992027 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwtn\" (UniqueName: \"kubernetes.io/projected/c35eab54-26ba-4fd5-958f-1e69cb47aae0-kube-api-access-xmwtn\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.084664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.084503 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:28:02.084664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.084568 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:28:02.093140 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.093111 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwtn\" (UniqueName: \"kubernetes.io/projected/c35eab54-26ba-4fd5-958f-1e69cb47aae0-kube-api-access-xmwtn\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.093285 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.093162 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-dshm\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.093285 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.093234 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-home\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.093285 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.093255 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-model-cache\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.093285 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.093275 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tls-certs\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.093458 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.093294 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tmp-dir\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.093685 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.093661 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tmp-dir\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.093937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.093906 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-model-cache\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.094055 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.094036 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-home\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.095820 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.095798 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-dshm\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.096741 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.096720 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tls-certs\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.104763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.104737 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwtn\" (UniqueName: \"kubernetes.io/projected/c35eab54-26ba-4fd5-958f-1e69cb47aae0-kube-api-access-xmwtn\") pod \"simulator-w1-6cf459-kserve-649fc74bcc-xq7ln\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.210137 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.210058 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:02.476788 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.469551 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:02.597344 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.596965 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln"] Oct 30 19:28:02.599004 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:28:02.598979 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc35eab54_26ba_4fd5_958f_1e69cb47aae0.slice/crio-9368c068f9efe70aed6061c5e0b00b0f88d1942f1a3ffbcc3053d9a4962dc347 WatchSource:0}: Error finding container 9368c068f9efe70aed6061c5e0b00b0f88d1942f1a3ffbcc3053d9a4962dc347: Status 404 returned error can't find the container with id 9368c068f9efe70aed6061c5e0b00b0f88d1942f1a3ffbcc3053d9a4962dc347 Oct 30 19:28:02.986791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.986663 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:28:02.986791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:02.986723 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:28:03.084779 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.084746 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:28:03.084991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.084816 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:28:03.339728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.338125 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:03.471376 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.471338 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:03.590837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.590754 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" event={"ID":"c35eab54-26ba-4fd5-958f-1e69cb47aae0","Type":"ContainerStarted","Data":"e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb"} Oct 30 19:28:03.590837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.590794 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" event={"ID":"c35eab54-26ba-4fd5-958f-1e69cb47aae0","Type":"ContainerStarted","Data":"9368c068f9efe70aed6061c5e0b00b0f88d1942f1a3ffbcc3053d9a4962dc347"} Oct 30 19:28:03.610981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.610939 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podStartSLOduration=2.6109262600000003 podStartE2EDuration="2.61092626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:03.609527826 +0000 UTC m=+504.336345123" watchObservedRunningTime="2026-10-30 19:28:03.61092626 +0000 UTC m=+504.337743557" Oct 30 19:28:03.986740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.986635 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:28:03.986740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:03.986688 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:28:04.084633 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:04.084599 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Oct 30 19:28:04.084838 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:04.084649 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Oct 30 19:28:04.210643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:04.210605 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:04.295323 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:04.295258 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:04.986916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:04.986882 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Oct 30 19:28:04.987333 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:04.986942 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Oct 30 19:28:05.114352 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:05.114323 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:28:05.122968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:05.122936 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:28:05.270919 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:05.263144 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:06.100496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:06.100462 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:28:06.101230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:06.101208 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:28:06.102057 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:06.102037 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:28:06.306208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:06.306172 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:07.106659 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:07.106617 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:07.863929 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:07.863897 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:08.586995 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:08.586955 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:09.313191 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.313158 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:09.490248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.490206 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r"] Oct 30 19:28:09.494572 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.494548 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.504471 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.504445 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"openshift-service-ca.crt\"" Oct 30 19:28:09.504648 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.504484 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"default-dockercfg-7qwvv\"" Oct 30 19:28:09.504781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.504617 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"simulator-w5-f7f075-kserve-self-signed-certs\"" Oct 30 19:28:09.504855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.504691 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"kube-root-ca.crt\"" Oct 30 19:28:09.506627 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.506600 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r"] Oct 30 19:28:09.593417 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.593309 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:09.603915 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.603886 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tmp-dir\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.604067 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.603987 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-dshm\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.604067 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.604032 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-model-cache\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.604163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.604088 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tls-certs\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.604163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.604117 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85tn5\" (UniqueName: \"kubernetes.io/projected/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-kube-api-access-85tn5\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.604163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.604154 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-home\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705102 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705066 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tmp-dir\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705123 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-dshm\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705151 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-model-cache\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705177 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tls-certs\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705197 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85tn5\" (UniqueName: \"kubernetes.io/projected/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-kube-api-access-85tn5\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705247 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-home\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705605 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705568 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tmp-dir\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705670 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705631 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-model-cache\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.705787 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.705764 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-home\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.707564 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.707542 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-dshm\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.707964 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.707944 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tls-certs\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.716537 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.716510 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85tn5\" (UniqueName: \"kubernetes.io/projected/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-kube-api-access-85tn5\") pod \"simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:09.811513 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:09.811478 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:10.088418 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:10.088394 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r"] Oct 30 19:28:10.090901 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:28:10.090859 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57d48892_90cc_4a09_80f8_3bd0d2c3a7b3.slice/crio-c85efe0529b67010a7948d100ba5db62cef8710ce0e9c34cb5b32948a33d38fc WatchSource:0}: Error finding container c85efe0529b67010a7948d100ba5db62cef8710ce0e9c34cb5b32948a33d38fc: Status 404 returned error can't find the container with id c85efe0529b67010a7948d100ba5db62cef8710ce0e9c34cb5b32948a33d38fc Oct 30 19:28:10.216385 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:10.216356 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:10.406394 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:28:10.406290 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:28:10.610423 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:10.610395 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:10.630177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:10.630144 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" event={"ID":"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3","Type":"ContainerStarted","Data":"0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143"} Oct 30 19:28:10.630177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:10.630180 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" event={"ID":"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3","Type":"ContainerStarted","Data":"c85efe0529b67010a7948d100ba5db62cef8710ce0e9c34cb5b32948a33d38fc"} Oct 30 19:28:10.648931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:10.648882 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podStartSLOduration=1.6488683339999999 podStartE2EDuration="1.648868334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:10.647849659 +0000 UTC m=+511.374666958" watchObservedRunningTime="2026-10-30 19:28:10.648868334 +0000 UTC m=+511.375685631" Oct 30 19:28:10.811949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:10.811914 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:11.187204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:11.187119 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:11.940415 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:11.940378 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:12.061151 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:12.061119 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:12.210855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:12.210769 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:12.223132 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:12.223109 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:12.650277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:12.650251 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:28:12.855817 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:12.855781 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:13.621072 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:13.621031 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:14.335745 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:14.335685 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:15.152345 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:15.152308 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:16.131272 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:16.131235 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:16.962230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:16.962168 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:17.827237 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:17.827210 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:19.811956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:19.811926 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:19.824526 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:19.824502 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:19.839600 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:19.839567 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:20.425441 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:20.425396 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:20.692940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:20.692862 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:28:21.121914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:21.121883 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:21.915116 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:21.915066 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:22.906010 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:22.905983 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:23.824053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:23.824008 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:24.515098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:24.515057 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:25.328888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:25.328849 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:25.628650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:25.628557 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:26.261150 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:26.261110 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:27.140636 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:27.140597 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:27.822961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:27.822928 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:28.714450 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.714413 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:28.768437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.768401 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4"] Oct 30 19:28:28.772173 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.772153 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:28.794317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.794294 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"kube-root-ca.crt\"" Oct 30 19:28:28.794489 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.794337 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"openshift-service-ca.crt\"" Oct 30 19:28:28.794489 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.794375 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"default-dockercfg-bhsf7\"" Oct 30 19:28:28.794489 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.794413 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"premium-w4-3bc085-kserve-self-signed-certs\"" Oct 30 19:28:28.797697 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.797667 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4"] Oct 30 19:28:28.840032 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.840005 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:28.913294 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.913256 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-dshm\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:28.913483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.913397 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-tmp-dir\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:28.913483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.913425 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ea159b5-d465-4a12-83ec-ad86afa91fec-tls-certs\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:28.913616 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.913566 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btgtq\" (UniqueName: \"kubernetes.io/projected/8ea159b5-d465-4a12-83ec-ad86afa91fec-kube-api-access-btgtq\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:28.913660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.913619 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-home\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:28.913693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:28.913671 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-model-cache\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015072 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015035 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-tmp-dir\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015242 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015123 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ea159b5-d465-4a12-83ec-ad86afa91fec-tls-certs\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015242 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015190 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btgtq\" (UniqueName: \"kubernetes.io/projected/8ea159b5-d465-4a12-83ec-ad86afa91fec-kube-api-access-btgtq\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015242 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015225 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-home\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015262 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-model-cache\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015304 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-dshm\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015560 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015527 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-tmp-dir\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015686 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-home\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.015730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.015721 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-model-cache\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.017975 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.017955 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-dshm\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.018062 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.017984 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ea159b5-d465-4a12-83ec-ad86afa91fec-tls-certs\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.025483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.025457 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btgtq\" (UniqueName: \"kubernetes.io/projected/8ea159b5-d465-4a12-83ec-ad86afa91fec-kube-api-access-btgtq\") pod \"premium-w4-3bc085-kserve-797d79d64f-nr8h4\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.084505 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.084471 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:29.360248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.360225 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4"] Oct 30 19:28:29.362456 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:28:29.362424 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ea159b5_d465_4a12_83ec_ad86afa91fec.slice/crio-0f2dabc358460a9799ab6979729f43c3be761a3a647542474fbbd8f4d7df4390 WatchSource:0}: Error finding container 0f2dabc358460a9799ab6979729f43c3be761a3a647542474fbbd8f4d7df4390: Status 404 returned error can't find the container with id 0f2dabc358460a9799ab6979729f43c3be761a3a647542474fbbd8f4d7df4390 Oct 30 19:28:29.437042 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.437000 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:29.565189 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.565159 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:29.733812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.733697 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" event={"ID":"8ea159b5-d465-4a12-83ec-ad86afa91fec","Type":"ContainerStarted","Data":"227ceed5896dbc42feb464dd9c958a10882535acf86ce3578716bce11661112f"} Oct 30 19:28:29.733812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.733779 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" event={"ID":"8ea159b5-d465-4a12-83ec-ad86afa91fec","Type":"ContainerStarted","Data":"0f2dabc358460a9799ab6979729f43c3be761a3a647542474fbbd8f4d7df4390"} Oct 30 19:28:29.756566 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:29.756513 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podStartSLOduration=1.7564951340000001 podStartE2EDuration="1.756495134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:29.753330897 +0000 UTC m=+530.480148193" watchObservedRunningTime="2026-10-30 19:28:29.756495134 +0000 UTC m=+530.483312431" Oct 30 19:28:30.084919 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:30.084879 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:30.629935 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:30.629905 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:31.520238 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:31.520208 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:31.859400 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:31.859314 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:32.008300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:32.008270 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:32.407544 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:32.407508 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:33.432809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:33.432779 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:34.009671 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.009636 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x"] Oct 30 19:28:34.013291 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.013269 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.025432 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.025406 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x"] Oct 30 19:28:34.028554 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.028518 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"premium-w5-f7f075-kserve-self-signed-certs\"" Oct 30 19:28:34.068957 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.068931 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tls-certs\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.069095 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.068972 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-dshm\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.069095 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.068992 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-model-cache\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.069186 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.069135 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp6ng\" (UniqueName: \"kubernetes.io/projected/09f38231-eeaf-48ad-b3d4-cfe1d497418c-kube-api-access-gp6ng\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.069186 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.069171 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tmp-dir\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.069291 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.069209 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-home\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.079037 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.079013 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:34.170617 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.170575 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-dshm\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.170617 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.170615 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-model-cache\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.170895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.170726 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp6ng\" (UniqueName: \"kubernetes.io/projected/09f38231-eeaf-48ad-b3d4-cfe1d497418c-kube-api-access-gp6ng\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.170895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.170779 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tmp-dir\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.170895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.170835 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-home\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.171052 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.170905 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tls-certs\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.171316 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.171282 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-model-cache\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.171457 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.171323 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tmp-dir\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.171457 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.171332 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-home\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.173202 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.173175 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-dshm\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.173594 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.173565 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tls-certs\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.185329 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.185305 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp6ng\" (UniqueName: \"kubernetes.io/projected/09f38231-eeaf-48ad-b3d4-cfe1d497418c-kube-api-access-gp6ng\") pod \"premium-w5-f7f075-kserve-5dbfc955fd-nkz4x\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.326089 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.326013 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:34.611203 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.611175 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x"] Oct 30 19:28:34.612745 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:28:34.612697 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09f38231_eeaf_48ad_b3d4_cfe1d497418c.slice/crio-e7365037f21a937fce7f91dbc366973ce8251a9ee6fd6cccb2d882fa57203621 WatchSource:0}: Error finding container e7365037f21a937fce7f91dbc366973ce8251a9ee6fd6cccb2d882fa57203621: Status 404 returned error can't find the container with id e7365037f21a937fce7f91dbc366973ce8251a9ee6fd6cccb2d882fa57203621 Oct 30 19:28:34.661356 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.661328 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:34.761066 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.761034 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" event={"ID":"09f38231-eeaf-48ad-b3d4-cfe1d497418c","Type":"ContainerStarted","Data":"4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4"} Oct 30 19:28:34.761204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.761073 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" event={"ID":"09f38231-eeaf-48ad-b3d4-cfe1d497418c","Type":"ContainerStarted","Data":"e7365037f21a937fce7f91dbc366973ce8251a9ee6fd6cccb2d882fa57203621"} Oct 30 19:28:34.791956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:34.791905 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podStartSLOduration=1.7918898410000001 podStartE2EDuration="1.791889841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:34.790577079 +0000 UTC m=+535.517394386" watchObservedRunningTime="2026-10-30 19:28:34.791889841 +0000 UTC m=+535.518707138" Oct 30 19:28:35.327142 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:35.327110 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:35.603935 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:35.603842 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:36.812863 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:36.812826 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:37.607972 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:37.607928 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:38.564987 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:38.564947 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:39.084797 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:39.084765 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:39.099509 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:39.099478 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:39.726265 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:39.726219 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:39.799973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:39.799946 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:28:40.404729 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:28:40.404658 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:28:40.847065 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:40.847029 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:41.798064 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:41.798026 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:42.692130 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:42.692100 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:43.688988 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:43.688951 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:44.137473 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.137441 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb"] Oct 30 19:28:44.142443 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.142427 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.151232 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.151209 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb"] Oct 30 19:28:44.160534 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.160511 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"premium-w6-639c15-kserve-self-signed-certs\"" Oct 30 19:28:44.160652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.160549 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"default-dockercfg-jxglj\"" Oct 30 19:28:44.160652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.160627 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"kube-root-ca.crt\"" Oct 30 19:28:44.160652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.160583 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"openshift-service-ca.crt\"" Oct 30 19:28:44.179680 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.179653 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-home\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.179789 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.179694 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-dshm\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.179789 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.179745 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2127368-191b-4224-83c9-b43f8b95e582-tls-certs\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.179873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.179846 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhr86\" (UniqueName: \"kubernetes.io/projected/f2127368-191b-4224-83c9-b43f8b95e582-kube-api-access-nhr86\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.179949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.179934 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-tmp-dir\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.180039 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.180022 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-model-cache\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.207754 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.207696 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:44.281034 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281005 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-model-cache\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.281190 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281104 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-home\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.281190 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281170 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-dshm\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.281309 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281196 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2127368-191b-4224-83c9-b43f8b95e582-tls-certs\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.281309 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281247 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhr86\" (UniqueName: \"kubernetes.io/projected/f2127368-191b-4224-83c9-b43f8b95e582-kube-api-access-nhr86\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.281430 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281342 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-tmp-dir\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.281625 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281603 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-home\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.281751 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281731 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-model-cache\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.281833 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.281783 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-tmp-dir\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.284887 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.284861 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-dshm\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.285279 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.285244 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2127368-191b-4224-83c9-b43f8b95e582-tls-certs\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.291425 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.291398 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhr86\" (UniqueName: \"kubernetes.io/projected/f2127368-191b-4224-83c9-b43f8b95e582-kube-api-access-nhr86\") pod \"premium-w6-639c15-kserve-65f5bc9df9-p9dwb\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.326508 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.326433 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:44.340377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.340352 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:44.455448 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.455369 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:44.733523 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.733496 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb"] Oct 30 19:28:44.735509 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:28:44.735479 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2127368_191b_4224_83c9_b43f8b95e582.slice/crio-02e6d76ad7f961c3de845d43d829adcf97f0c65b760677c4770db60ecf4388d8 WatchSource:0}: Error finding container 02e6d76ad7f961c3de845d43d829adcf97f0c65b760677c4770db60ecf4388d8: Status 404 returned error can't find the container with id 02e6d76ad7f961c3de845d43d829adcf97f0c65b760677c4770db60ecf4388d8 Oct 30 19:28:44.748033 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.747997 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:44.816856 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.816824 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" event={"ID":"f2127368-191b-4224-83c9-b43f8b95e582","Type":"ContainerStarted","Data":"b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69"} Oct 30 19:28:44.816965 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.816876 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" event={"ID":"f2127368-191b-4224-83c9-b43f8b95e582","Type":"ContainerStarted","Data":"02e6d76ad7f961c3de845d43d829adcf97f0c65b760677c4770db60ecf4388d8"} Oct 30 19:28:44.828528 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.828504 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:28:44.840072 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.840032 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podStartSLOduration=0.84002003 podStartE2EDuration="840.02003ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:44.838351069 +0000 UTC m=+545.565168375" watchObservedRunningTime="2026-10-30 19:28:44.84002003 +0000 UTC m=+545.566837327" Oct 30 19:28:44.897597 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:44.897561 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:45.456205 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:45.456177 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:45.651984 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:45.651955 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:46.785420 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:46.785382 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:47.966490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:47.966458 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:48.100081 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:48.100038 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:49.050994 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:49.050955 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:50.245357 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:50.245072 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:51.152061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:51.152021 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:52.792377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:52.792341 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:53.805256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:53.805226 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:54.455766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:54.455730 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:54.468545 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:54.468517 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:54.611479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:54.611431 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:54.885306 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:54.885278 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:28:55.564617 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:55.564584 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:56.534496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:56.534462 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:57.636610 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:57.636574 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:58.657864 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:58.657831 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:59.696757 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:28:59.696723 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:00.536024 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:00.535995 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:01.365419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:01.365386 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:02.329337 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:02.329302 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:03.403530 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:03.403501 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:04.333224 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:04.333165 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:05.332122 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:05.332085 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:06.209236 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:06.209193 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:07.056479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:07.056447 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:07.925008 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:07.924971 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:09.100065 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:09.100025 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:09.797382 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:09.797339 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:10.404121 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:29:10.404088 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:29:10.604088 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:10.604040 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:11.389590 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:11.389535 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:12.190603 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:12.190561 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:13.095352 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:13.095308 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:13.820434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:13.820384 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:14.606447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:14.606407 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:15.326019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:15.325977 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:16.024428 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:16.024398 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:16.818409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:16.818372 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:17.669108 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:17.669075 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:18.409083 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:18.409038 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:19.171669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:19.171637 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:19.904857 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:19.904823 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:20.697957 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:20.697923 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:21.416647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:21.416610 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:22.130729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:22.130672 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:22.800680 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:22.800632 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:23.526425 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:23.526392 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:24.539123 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:24.539090 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:25.192916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:25.192883 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:25.949880 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:25.949840 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:26.802122 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:26.801383 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:27.669579 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:27.669535 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:28.424483 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.424451 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg"] Oct 30 19:29:28.427822 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.427806 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.435123 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.435107 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-9f2d57-openshift-default-dockercfg-l4n9q\"" Oct 30 19:29:28.443011 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.442987 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg"] Oct 30 19:29:28.481362 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.481335 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.481504 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.481368 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.481504 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.481473 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.481641 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.481522 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.481811 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.481655 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7qb4\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-kube-api-access-j7qb4\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.481893 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.481846 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.481893 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.481872 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.482024 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.482005 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.482086 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.482053 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.582761 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.582729 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.582937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.582772 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.582937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.582831 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.582937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.582848 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.582937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.582898 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583225 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.582939 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583225 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.582977 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7qb4\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-kube-api-access-j7qb4\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583225 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.583026 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583225 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.583051 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.583391 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583599 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.583574 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583696 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.583636 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.583732 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.583780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.583748 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.585888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.585865 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.586099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.586079 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.594395 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.594367 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.594497 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.594480 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7qb4\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-kube-api-access-j7qb4\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.740936 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.740910 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:28.742606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:28.742584 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:29.037627 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:29.037601 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg"] Oct 30 19:29:29.038769 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:29:29.038735 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be4340f_16bb_46ad_8117_4d31a32aa2c1.slice/crio-ebdecf8f1fbd45ba19c18819a6cfe842e7333b73ca338190c02e885dc392877a WatchSource:0}: Error finding container ebdecf8f1fbd45ba19c18819a6cfe842e7333b73ca338190c02e885dc392877a: Status 404 returned error can't find the container with id ebdecf8f1fbd45ba19c18819a6cfe842e7333b73ca338190c02e885dc392877a Oct 30 19:29:29.041210 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:29.041181 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:29.041295 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:29.041245 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:29.041295 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:29.041273 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:29.081442 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:29.081411 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" event={"ID":"5be4340f-16bb-46ad-8117-4d31a32aa2c1","Type":"ContainerStarted","Data":"ebdecf8f1fbd45ba19c18819a6cfe842e7333b73ca338190c02e885dc392877a"} Oct 30 19:29:29.519133 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:29.519098 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:30.109143 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:30.109114 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" event={"ID":"5be4340f-16bb-46ad-8117-4d31a32aa2c1","Type":"ContainerStarted","Data":"f8a6670d890202195ef26b53d96e6304d480239b8402ff47ba11c1446b02c869"} Oct 30 19:29:30.155130 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:30.155065 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" podStartSLOduration=2.155044898 podStartE2EDuration="2.155044898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:30.137161328 +0000 UTC m=+590.863978960" watchObservedRunningTime="2026-10-30 19:29:30.155044898 +0000 UTC m=+590.881862197" Oct 30 19:29:30.498454 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:30.498400 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:30.757776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:30.749635 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:30.762432 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:30.759835 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:29:30.762432 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:30.759913 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:29:31.215346 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:31.215270 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:31.742128 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:31.742093 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:29:31.742316 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:31.742153 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:29:32.009908 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:32.009798 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:32.746121 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:32.746087 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:29:32.746740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:32.746685 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:29:32.825750 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:32.824617 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:33.542379 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:33.542349 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:33.741406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:33.741372 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:29:33.741627 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:33.741433 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:29:34.288968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:34.288929 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:34.743100 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:34.743064 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:29:34.743296 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:34.743127 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:29:35.113045 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:35.112960 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:35.718344 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:35.718307 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:35.916634 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:35.916602 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:35.917587 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:35.917552 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:35.918398 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:35.918379 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:29:36.511970 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:36.511940 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:37.295313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:37.295282 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:38.061772 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:38.061728 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:38.958113 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:38.958079 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:39.448437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:39.448406 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:29:39.449461 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:39.449429 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:29:39.799188 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:39.799148 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:40.403383 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:29:40.403347 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:29:40.933987 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:40.933953 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:41.833025 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:41.832995 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:42.617154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.617118 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:42.840649 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.840621 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r"] Oct 30 19:29:42.845146 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.845114 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.854047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.854028 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-3bc085-openshift-default-dockercfg-6wbgq\"" Oct 30 19:29:42.859354 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.859330 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r"] Oct 30 19:29:42.986018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.985938 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231c554b-45d4-41d7-9f86-5b2255306400-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.986018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.985975 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.986018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.985998 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.986255 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.986064 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231c554b-45d4-41d7-9f86-5b2255306400-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.986255 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.986133 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8kkd\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-kube-api-access-k8kkd\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.986255 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.986169 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.986255 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.986235 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.986435 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.986267 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:42.986435 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:42.986406 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087140 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087106 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8kkd\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-kube-api-access-k8kkd\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087140 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087145 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087166 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087283 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087471 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087427 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087531 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087496 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231c554b-45d4-41d7-9f86-5b2255306400-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087593 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087534 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087593 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087544 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087593 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087572 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087765 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087597 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231c554b-45d4-41d7-9f86-5b2255306400-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087765 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087667 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087776 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.087949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.087929 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.088146 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.088126 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231c554b-45d4-41d7-9f86-5b2255306400-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.090031 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.090008 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.090201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.090181 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231c554b-45d4-41d7-9f86-5b2255306400-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.104209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.104184 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8kkd\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-kube-api-access-k8kkd\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.111718 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.111685 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.136557 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.136532 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2"] Oct 30 19:29:43.140984 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.140969 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.151673 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.151651 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-171808-openshift-default-dockercfg-mzssz\"" Oct 30 19:29:43.154936 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.154915 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2"] Oct 30 19:29:43.159637 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.159618 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:43.289565 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.289518 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.289737 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.289577 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zjtm\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-kube-api-access-4zjtm\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.289793 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.289770 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.289829 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.289802 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe1332a-d689-4113-a3a5-b16e82f1f167-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.289902 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.289884 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.290017 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.289995 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.290062 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.290050 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.290100 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.290073 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.290139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.290121 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.390858 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.390828 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.390964 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.390872 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.390964 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.390898 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.390964 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.390955 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.391131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.390986 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.391131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.391015 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zjtm\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-kube-api-access-4zjtm\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.391131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.391090 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.391131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.391124 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe1332a-d689-4113-a3a5-b16e82f1f167-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.391316 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.391176 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.391515 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.391482 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.391697 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.391657 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.391956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.391922 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.392049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.391991 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.392136 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.392114 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe1332a-d689-4113-a3a5-b16e82f1f167-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.393467 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.393447 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.393731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.393696 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.404437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.404411 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.404862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.404840 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zjtm\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-kube-api-access-4zjtm\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.454538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.454502 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:43.454835 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.454813 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r"] Oct 30 19:29:43.456075 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:29:43.456045 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231c554b_45d4_41d7_9f86_5b2255306400.slice/crio-f6c9dfc061a4722f5ea006d11fd5f8c9593485505f548958049928b334cd95a5 WatchSource:0}: Error finding container f6c9dfc061a4722f5ea006d11fd5f8c9593485505f548958049928b334cd95a5: Status 404 returned error can't find the container with id f6c9dfc061a4722f5ea006d11fd5f8c9593485505f548958049928b334cd95a5 Oct 30 19:29:43.456646 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.456615 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:43.459021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.458991 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:43.459117 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.459055 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:43.459117 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.459082 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:43.748697 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.748668 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2"] Oct 30 19:29:43.749666 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:29:43.749638 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe1332a_d689_4113_a3a5_b16e82f1f167.slice/crio-305b2821e3b69425c0ed2189e766cb2034e3fcd4ef6916ff99dfbf6d21a901c1 WatchSource:0}: Error finding container 305b2821e3b69425c0ed2189e766cb2034e3fcd4ef6916ff99dfbf6d21a901c1: Status 404 returned error can't find the container with id 305b2821e3b69425c0ed2189e766cb2034e3fcd4ef6916ff99dfbf6d21a901c1 Oct 30 19:29:43.752662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.752618 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:43.752770 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.752695 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:43.752770 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:43.752752 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:44.211378 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.211286 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" event={"ID":"231c554b-45d4-41d7-9f86-5b2255306400","Type":"ContainerStarted","Data":"753594791f9e05a2158d446691afa3b01b049bfad17ba977e3dd084ef1a50d55"} Oct 30 19:29:44.211378 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.211348 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" event={"ID":"231c554b-45d4-41d7-9f86-5b2255306400","Type":"ContainerStarted","Data":"f6c9dfc061a4722f5ea006d11fd5f8c9593485505f548958049928b334cd95a5"} Oct 30 19:29:44.212980 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.212954 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" event={"ID":"fbe1332a-d689-4113-a3a5-b16e82f1f167","Type":"ContainerStarted","Data":"48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7"} Oct 30 19:29:44.213086 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.213007 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" event={"ID":"fbe1332a-d689-4113-a3a5-b16e82f1f167","Type":"ContainerStarted","Data":"305b2821e3b69425c0ed2189e766cb2034e3fcd4ef6916ff99dfbf6d21a901c1"} Oct 30 19:29:44.245726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.245151 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podStartSLOduration=2.245133394 podStartE2EDuration="2.245133394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:44.244377688 +0000 UTC m=+604.971194986" watchObservedRunningTime="2026-10-30 19:29:44.245133394 +0000 UTC m=+604.971950691" Oct 30 19:29:44.284729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.283194 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podStartSLOduration=1.283177831 podStartE2EDuration="1.283177831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:44.282442098 +0000 UTC m=+605.009259408" watchObservedRunningTime="2026-10-30 19:29:44.283177831 +0000 UTC m=+605.009995128" Oct 30 19:29:44.362728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.359055 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:44.456901 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.456871 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:44.461902 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.461816 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Oct 30 19:29:44.462139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.462113 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Oct 30 19:29:44.953734 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:44.953621 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:45.166826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:45.166794 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:45.167599 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:45.167569 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:29:45.167760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:45.167648 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:29:45.261776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:45.260799 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:45.462728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:45.462097 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Oct 30 19:29:45.462728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:45.462156 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Oct 30 19:29:46.041729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:46.038969 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:46.164728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:46.163958 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:29:46.164728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:46.164019 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:29:46.463783 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:46.460165 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Oct 30 19:29:46.463783 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:46.460221 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Oct 30 19:29:47.140727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:47.140375 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:47.163729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:47.160112 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:29:47.163729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:47.160182 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:29:47.455791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:47.455219 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Oct 30 19:29:47.455791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:47.455284 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Oct 30 19:29:48.163729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:48.163092 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:29:48.163729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:48.163155 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:29:48.341149 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:48.341113 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:48.469283 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:48.458178 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Oct 30 19:29:48.469283 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:48.458240 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Oct 30 19:29:49.165164 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:49.165125 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:29:49.165806 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:49.165770 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:29:49.457554 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:49.457447 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Oct 30 19:29:49.457869 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:49.457814 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Oct 30 19:29:49.745728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:49.743245 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:50.160901 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:50.160813 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:29:50.160901 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:50.160884 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:29:50.607681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:50.607647 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:51.445340 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.445308 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:51.498862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.498827 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:29:51.499026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.498894 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:29:51.582989 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.582962 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:51.583471 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.583447 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:51.584828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.584808 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:29:51.594030 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.594010 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:51.594481 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.594454 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:51.595596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:51.595572 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:29:52.627893 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:52.627859 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:52.752350 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:52.752319 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:53.589861 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:53.589831 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:54.593265 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:54.593228 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:55.415723 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:55.415653 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:56.333226 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:56.333196 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:57.215923 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:57.215889 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:58.411731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.409873 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf"] Oct 30 19:29:58.414509 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.414481 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.434938 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.434121 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-6cf459-openshift-default-dockercfg-ckp8r\"" Oct 30 19:29:58.435730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.435138 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf"] Oct 30 19:29:58.543571 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.543527 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.543954 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.543911 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751225-aab9-4454-ab45-f9ace259a4ff-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.544233 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.544205 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.544441 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.544423 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.545424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.545401 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.545649 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.545631 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.550649 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.550628 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.570168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.570135 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.570354 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.570334 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfnhd\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-kube-api-access-vfnhd\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.587960 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.587931 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:58.685948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.675327 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.686477 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.686452 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.687821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.687786 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.687983 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.687962 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.688204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.688183 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.688365 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.688345 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfnhd\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-kube-api-access-vfnhd\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.688550 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.688537 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.688719 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.688686 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751225-aab9-4454-ab45-f9ace259a4ff-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.688904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.688886 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.689392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.689373 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.689537 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.687036 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.690437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.690407 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.690844 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.690825 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.694731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.691163 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751225-aab9-4454-ab45-f9ace259a4ff-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.722747 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.716677 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.722747 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.716961 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.722747 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.719798 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.750192 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.750146 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfnhd\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-kube-api-access-vfnhd\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:58.792795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:58.792094 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:29:59.122910 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:59.122881 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf"] Oct 30 19:29:59.124056 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:29:59.124023 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca751225_aab9_4454_ab45_f9ace259a4ff.slice/crio-9c205183dd546b2ae8b748c1eb71af9a1fed247a2be5fb3349587a4802a93f27 WatchSource:0}: Error finding container 9c205183dd546b2ae8b748c1eb71af9a1fed247a2be5fb3349587a4802a93f27: Status 404 returned error can't find the container with id 9c205183dd546b2ae8b748c1eb71af9a1fed247a2be5fb3349587a4802a93f27 Oct 30 19:29:59.127469 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:59.127426 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:59.127576 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:59.127522 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:59.127649 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:59.127570 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:29:59.569730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:59.569062 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" event={"ID":"ca751225-aab9-4454-ab45-f9ace259a4ff","Type":"ContainerStarted","Data":"631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242"} Oct 30 19:29:59.569730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:59.569136 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" event={"ID":"ca751225-aab9-4454-ab45-f9ace259a4ff","Type":"ContainerStarted","Data":"9c205183dd546b2ae8b748c1eb71af9a1fed247a2be5fb3349587a4802a93f27"} Oct 30 19:29:59.794872 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:29:59.794782 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:30:00.796752 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:00.796302 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 19:30:00.796752 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:00.796367 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 19:30:01.371072 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:01.371028 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:01.459846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:01.459806 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:30:01.460127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:01.460094 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:01.792427 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:01.792390 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 19:30:01.792596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:01.792450 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 19:30:02.793065 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:02.793031 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 19:30:02.793454 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:02.793094 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 19:30:03.603654 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.603615 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:03.694681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.694607 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" podStartSLOduration=5.69458836 podStartE2EDuration="5.69458836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:59.603731366 +0000 UTC m=+620.330548657" watchObservedRunningTime="2026-10-30 19:30:03.69458836 +0000 UTC m=+624.421405657" Oct 30 19:30:03.695830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.695802 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn"] Oct 30 19:30:03.700446 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.700423 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.710923 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.710901 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"default-dockercfg-jjz7n\"" Oct 30 19:30:03.712146 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.712116 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"distinct-w3-9f2d57-kserve-self-signed-certs\"" Oct 30 19:30:03.712272 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.712170 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"kube-root-ca.crt\"" Oct 30 19:30:03.712424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.712411 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"openshift-service-ca.crt\"" Oct 30 19:30:03.714794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.714777 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn"] Oct 30 19:30:03.789432 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.789404 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:03.792494 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.792448 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 19:30:03.792619 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.792513 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 19:30:03.797045 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.797018 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxszc\" (UniqueName: \"kubernetes.io/projected/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-kube-api-access-mxszc\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.797384 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.797080 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tmp-dir\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.797384 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.797197 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tls-certs\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.797384 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.797219 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-model-cache\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.797384 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.797256 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-home\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.797577 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.797395 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-dshm\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.898781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.898455 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tmp-dir\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.898781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.898540 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tls-certs\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.898781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.898574 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-model-cache\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.898781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.898629 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-home\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.898781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.898657 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-dshm\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.899193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.898829 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxszc\" (UniqueName: \"kubernetes.io/projected/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-kube-api-access-mxszc\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.914681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.899621 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tmp-dir\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.914681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.900755 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-home\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.914681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.900994 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-model-cache\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.952414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.952331 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tls-certs\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.955981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.954210 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxszc\" (UniqueName: \"kubernetes.io/projected/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-kube-api-access-mxszc\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:03.955981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:03.954387 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-dshm\") pod \"distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:04.041745 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:04.041339 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:04.346043 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:04.346021 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn"] Oct 30 19:30:04.348546 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:30:04.348518 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod534e4ef6_7f4f_46db_bf68_8ea9cbaf48f5.slice/crio-dd2d3b85074f81aeb419c55d16a5767696b76a783c5a648c8701041d2d1d9bc4 WatchSource:0}: Error finding container dd2d3b85074f81aeb419c55d16a5767696b76a783c5a648c8701041d2d1d9bc4: Status 404 returned error can't find the container with id dd2d3b85074f81aeb419c55d16a5767696b76a783c5a648c8701041d2d1d9bc4 Oct 30 19:30:04.658199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:04.658115 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" event={"ID":"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5","Type":"ContainerStarted","Data":"ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9"} Oct 30 19:30:04.658199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:04.658159 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" event={"ID":"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5","Type":"ContainerStarted","Data":"dd2d3b85074f81aeb419c55d16a5767696b76a783c5a648c8701041d2d1d9bc4"} Oct 30 19:30:04.682212 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:04.682169 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podStartSLOduration=1.682151919 podStartE2EDuration="1.682151919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:04.678598531 +0000 UTC m=+625.405415828" watchObservedRunningTime="2026-10-30 19:30:04.682151919 +0000 UTC m=+625.408969218" Oct 30 19:30:04.792499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:04.792467 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 19:30:04.792671 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:04.792530 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 19:30:04.872947 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:04.872909 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:05.043565 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:05.043537 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:05.792549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:05.792517 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Oct 30 19:30:05.792736 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:05.792567 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Oct 30 19:30:06.006914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:06.006882 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:06.919399 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:06.919339 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:06.934961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:06.934934 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:30:06.935463 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:06.935439 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:30:06.936438 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:06.936412 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:30:07.160148 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:07.160112 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:07.404305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:07.404268 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:08.208418 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:08.208380 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:09.026641 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:09.026602 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:09.833070 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:09.833024 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:10.403042 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:30:10.402998 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:30:10.559131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:10.559097 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:11.328192 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:11.328158 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:12.125402 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:12.125370 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:13.155567 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.155529 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:13.350974 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.350931 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8"] Oct 30 19:30:13.356081 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.356054 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.380408 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.380369 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8"] Oct 30 19:30:13.429774 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.429022 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd"] Oct 30 19:30:13.438726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.433958 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x"] Oct 30 19:30:13.438726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.437091 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.438932 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.438802 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.438932 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.438874 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.439040 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.438935 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.439040 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.438972 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.439040 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.439019 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.439193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.439046 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/568a6fd2-2a37-40cf-a335-be350847e9ea-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.439193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.439144 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlhr9\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-kube-api-access-zlhr9\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.439193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.439173 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.439356 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.439228 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.452747 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.442686 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.556981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.543156 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-d55178-openshift-default-dockercfg-6lspn\"" Oct 30 19:30:13.586869 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.586841 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd"] Oct 30 19:30:13.623561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.593771 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.623561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.597184 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x"] Oct 30 19:30:13.623561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.612792 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.623561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.613002 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.623561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.613057 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.640815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.613113 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.651814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.646828 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlhr9\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-kube-api-access-zlhr9\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.651814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.646893 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbbxl\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-kube-api-access-hbbxl\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.651814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.646931 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.655796 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.647025 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.656064 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.656022 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.660965 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661031 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661081 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661110 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661156 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661205 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661246 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661336 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661373 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661405 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661456 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7c7q\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-kube-api-access-x7c7q\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661502 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661541 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661572 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661623 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661653 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/568a6fd2-2a37-40cf-a335-be350847e9ea-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.661750 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.662742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.659659 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.663962 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.662757 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/568a6fd2-2a37-40cf-a335-be350847e9ea-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.663962 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.663297 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.663962 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.663532 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.714792 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.676940 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.714792 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.682204 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.714792 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.696069 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.714792 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.705246 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.730745 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.723735 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlhr9\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-kube-api-access-zlhr9\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.730745 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.724380 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:13.813933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.813875 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.813933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.813922 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.814163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.813959 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7c7q\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-kube-api-access-x7c7q\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.814163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814003 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.814163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814035 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.814163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814110 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.814163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814153 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.814437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814178 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.814437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814202 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.814437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814236 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.814437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814289 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.814437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814328 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbbxl\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-kube-api-access-hbbxl\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.814437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814403 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.814437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814435 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.814854 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.814505 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.816363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.816305 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.818200 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.817828 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.830539 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.830513 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.834959 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.831325 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.835098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.831892 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.835200 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.832137 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.835298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.833659 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.835394 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.833963 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.847232 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.845390 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.849379 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.849348 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.849491 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.849416 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.849491 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.849481 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.854096 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.853070 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.854096 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.853322 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.855527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.855483 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.919292 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.917179 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.919292 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.919237 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.919539 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.919521 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.931475 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.927395 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbbxl\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-kube-api-access-hbbxl\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.936646 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.936449 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7c7q\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-kube-api-access-x7c7q\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:13.970842 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.968360 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:13.970842 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:13.968423 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:14.043049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:14.042253 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:14.079288 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:14.078932 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:14.790849 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:14.790815 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8"] Oct 30 19:30:14.805539 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:14.803963 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:14.845728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:14.838788 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:14.850205 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:30:14.849162 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568a6fd2_2a37_40cf_a335_be350847e9ea.slice/crio-076a1278d4d5bc2b8bf274e5e4634a9b19c4d40f733256fb82dcb356934b3bed WatchSource:0}: Error finding container 076a1278d4d5bc2b8bf274e5e4634a9b19c4d40f733256fb82dcb356934b3bed: Status 404 returned error can't find the container with id 076a1278d4d5bc2b8bf274e5e4634a9b19c4d40f733256fb82dcb356934b3bed Oct 30 19:30:14.939728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:14.918901 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:30:15.049749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.049630 5627 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:15.062726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.051640 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:15.094339 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.065593 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:15.094339 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.074921 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:15.094339 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.074980 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:15.648247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.647614 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:15.732155 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.710817 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:15.732155 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.710926 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": context deadline exceeded" Oct 30 19:30:15.732155 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.712045 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:15.830728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.805349 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:15.852730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.835798 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:30:15.935474 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.935371 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:15.979080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.936567 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded" Oct 30 19:30:16.084251 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.957385 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:16.157756 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:15.980135 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:30:16.157756 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.125739 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:30:16.502452 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.435124 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x"] Oct 30 19:30:16.502452 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.497054 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:16.502452 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.497454 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:16.502452 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.497892 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:16.502452 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.500086 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" event={"ID":"568a6fd2-2a37-40cf-a335-be350847e9ea","Type":"ContainerStarted","Data":"076a1278d4d5bc2b8bf274e5e4634a9b19c4d40f733256fb82dcb356934b3bed"} Oct 30 19:30:16.556733 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.542836 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:30:16.630764 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.581241 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:16.677805 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.580969 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:30:16.678102 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.678066 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:16.737571 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.737507 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:16.749925 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.749791 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:30:16.847778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.847351 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:16.886757 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:16.876773 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:17.007729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.003849 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:17.193438 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.158828 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:17.193739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.160226 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:17.194370 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.194346 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:30:17.357799 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.319299 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:17.409980 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.409925 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:17.410282 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.410256 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:17.497738 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.483150 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd"] Oct 30 19:30:17.598242 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.598216 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:30:17.703794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.679322 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:30:17.751026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.714222 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:30:17.804833 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.792875 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:17.807128 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.807086 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:30:17.873826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.873472 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" event={"ID":"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3","Type":"ContainerStarted","Data":"4cbd702bac2d46b850d269b5bf6649fcdb928f41dbdd7f8d6aa004451d9d186d"} Oct 30 19:30:17.915498 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.915439 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" event={"ID":"49657d5c-b0b6-4abd-9a6f-dc8b3071e212","Type":"ContainerStarted","Data":"8d4c381ce9b9e36f1f871427fc8dc29efc07b181dacc1f0817cd1fec2d6c5942"} Oct 30 19:30:17.924888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.924853 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" event={"ID":"568a6fd2-2a37-40cf-a335-be350847e9ea","Type":"ContainerStarted","Data":"66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a"} Oct 30 19:30:17.991131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:17.991100 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:30:18.014988 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:18.007125 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podStartSLOduration=5.007105357 podStartE2EDuration="5.007105357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:18.005127429 +0000 UTC m=+638.731944729" watchObservedRunningTime="2026-10-30 19:30:18.007105357 +0000 UTC m=+638.733922660" Oct 30 19:30:18.732811 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:18.732779 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:18.733992 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:18.733964 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 19:30:18.734224 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:18.734197 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 19:30:18.985915 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:18.985805 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" event={"ID":"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3","Type":"ContainerStarted","Data":"363497b64f8822ae25ab63dca0365c985be2a0960d75df6abaa5b02bb648b6c4"} Oct 30 19:30:18.990890 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:18.990858 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" event={"ID":"49657d5c-b0b6-4abd-9a6f-dc8b3071e212","Type":"ContainerStarted","Data":"690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0"} Oct 30 19:30:19.083268 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.083203 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podStartSLOduration=6.083184122 podStartE2EDuration="6.083184122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:19.041119401 +0000 UTC m=+639.767936698" watchObservedRunningTime="2026-10-30 19:30:19.083184122 +0000 UTC m=+639.810001420" Oct 30 19:30:19.086756 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.085302 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:19.087045 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.085371 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 19:30:19.087220 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.087199 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 19:30:19.098812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.088381 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podStartSLOduration=6.088372105 podStartE2EDuration="6.088372105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:19.087144028 +0000 UTC m=+639.813961320" watchObservedRunningTime="2026-10-30 19:30:19.088372105 +0000 UTC m=+639.815189391" Oct 30 19:30:19.111464 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.111433 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:19.739728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.739111 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 19:30:19.739728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.739169 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 19:30:19.984729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.984384 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 19:30:19.984729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.984461 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 19:30:19.984946 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:19.984836 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:20.081789 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:20.081106 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 19:30:20.081789 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:20.081164 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 19:30:20.388828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:20.379203 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:20.757790 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:20.754596 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 19:30:20.757790 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:20.754665 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 19:30:20.946758 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:20.946533 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:20.985128 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:20.985093 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 19:30:20.985916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:20.985882 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 19:30:21.093821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:21.092192 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 19:30:21.093821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:21.092270 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 19:30:21.766732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:21.763126 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 19:30:21.766732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:21.763187 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 19:30:21.766732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:21.763289 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:22.034788 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:21.991155 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 19:30:22.034788 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:21.991271 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 19:30:22.090732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:22.081155 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 19:30:22.090732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:22.081225 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 19:30:22.734283 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:22.734250 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 19:30:22.734984 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:22.734934 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 19:30:22.872551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:22.872517 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:22.972352 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:22.972309 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 19:30:22.972628 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:22.972602 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 19:30:23.086732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:23.083084 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 19:30:23.086732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:23.083145 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 19:30:23.785227 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:23.785078 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Oct 30 19:30:23.785227 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:23.785135 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Oct 30 19:30:23.791420 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:23.791381 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:23.973731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:23.970227 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 19:30:23.973731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:23.970302 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 19:30:24.080386 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.080290 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 19:30:24.080667 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.080634 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 19:30:24.088022 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.087999 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:24.099888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.099854 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:30:24.688536 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.688495 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:24.736499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.736461 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:24.736969 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.736953 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:24.737760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.737743 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:30:24.824320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.824290 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:24.970832 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.970735 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Oct 30 19:30:24.970832 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:24.970785 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Oct 30 19:30:25.079918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:25.079877 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Oct 30 19:30:25.080097 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:25.079940 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Oct 30 19:30:25.619625 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:25.619585 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:25.974477 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:25.974392 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:25.974928 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:25.974883 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:25.975578 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:25.975562 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:30:26.085490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:26.085460 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:26.086116 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:26.086096 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:26.087131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:26.087108 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:30:26.389560 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:26.389520 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:27.331722 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:27.331667 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:27.980129 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:27.980093 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m"] Oct 30 19:30:27.984903 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:27.984881 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.052804 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.052769 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.052995 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.052888 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.052995 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.052949 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.053119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.053096 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.053179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.053143 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.053234 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.053211 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.053289 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.053250 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jngrj\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-kube-api-access-jngrj\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.053389 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.053373 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.053459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.053429 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1893e40-3612-4b59-ad5b-ec8144073ea4-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.084311 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.084282 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m"] Oct 30 19:30:28.154743 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.154687 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.154898 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.154768 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.154898 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.154792 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.154898 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.154815 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.154898 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.154836 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jngrj\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-kube-api-access-jngrj\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155121 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.154932 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155121 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.155007 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1893e40-3612-4b59-ad5b-ec8144073ea4-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155121 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.155064 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155121 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.155109 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.155145 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.155280 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155462 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.155439 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155546 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.155479 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.155917 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.155898 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1893e40-3612-4b59-ad5b-ec8144073ea4-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.157302 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.157273 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.157518 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.157499 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.165524 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.165503 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jngrj\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-kube-api-access-jngrj\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.169207 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.169182 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.300652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.300623 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:28.314883 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.314857 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:28.591664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.591636 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m"] Oct 30 19:30:28.593572 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:30:28.593549 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1893e40_3612_4b59_ad5b_ec8144073ea4.slice/crio-a7741716ded86f25ad2d9190e33a0046e6e71bb6952696a7e8e62f10e5cf9ea0 WatchSource:0}: Error finding container a7741716ded86f25ad2d9190e33a0046e6e71bb6952696a7e8e62f10e5cf9ea0: Status 404 returned error can't find the container with id a7741716ded86f25ad2d9190e33a0046e6e71bb6952696a7e8e62f10e5cf9ea0 Oct 30 19:30:28.596025 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.595995 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:28.596094 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.596055 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:28.596094 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:28.596081 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:29.270286 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:29.270246 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" event={"ID":"e1893e40-3612-4b59-ad5b-ec8144073ea4","Type":"ContainerStarted","Data":"eb902bf9beeabf084b5221252bcc7559ce007c95417f5dea657333387b5fd414"} Oct 30 19:30:29.270650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:29.270629 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" event={"ID":"e1893e40-3612-4b59-ad5b-ec8144073ea4","Type":"ContainerStarted","Data":"a7741716ded86f25ad2d9190e33a0046e6e71bb6952696a7e8e62f10e5cf9ea0"} Oct 30 19:30:29.300775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:29.300744 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:29.302074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:29.302049 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 19:30:29.302168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:29.302120 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 19:30:29.308270 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:29.308228 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podStartSLOduration=2.308216841 podStartE2EDuration="2.308216841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:29.306518393 +0000 UTC m=+650.033335690" watchObservedRunningTime="2026-10-30 19:30:29.308216841 +0000 UTC m=+650.035034138" Oct 30 19:30:29.343189 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:29.343154 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:30.301795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:30.301760 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 19:30:30.302169 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:30.301826 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 19:30:30.453071 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:30.453040 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:31.301977 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:31.301944 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 19:30:31.302374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:31.302002 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 19:30:31.446174 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:31.446102 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:32.007441 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.007399 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv"] Oct 30 19:30:32.012224 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.012202 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.030110 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.030086 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 30 19:30:32.030323 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.030306 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 30 19:30:32.044726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.030513 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 30 19:30:32.045292 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.044310 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-t7289\"" Oct 30 19:30:32.049935 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.049909 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv"] Oct 30 19:30:32.100360 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.100333 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjhqq\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-kube-api-access-kjhqq\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.100495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.100397 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.100495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.100469 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.100620 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.100521 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b3c348-5b38-4a00-b541-36ecdda3b032-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.100620 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.100602 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.100733 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.100640 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.100781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.100762 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.201850 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.201816 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.201993 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.201921 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjhqq\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-kube-api-access-kjhqq\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.201993 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.201974 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.202106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.202009 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.202106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.202067 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b3c348-5b38-4a00-b541-36ecdda3b032-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.202215 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.202112 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.202215 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.202149 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.202821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.202770 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.204600 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.204572 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.204694 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.204605 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.204938 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.204917 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b3c348-5b38-4a00-b541-36ecdda3b032-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.205008 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.204974 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.212828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.212804 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.212920 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.212799 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjhqq\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-kube-api-access-kjhqq\") pod \"istiod-openshift-gateway-7ddd9cfdc8-mtqgv\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.301119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.301053 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 19:30:32.301240 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.301125 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 19:30:32.354491 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.354452 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:32.486929 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.486896 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:32.646077 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.645440 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv"] Oct 30 19:30:32.647890 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:30:32.647856 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b3c348_5b38_4a00_b541_36ecdda3b032.slice/crio-e2c63de1aab5663dbeb8089cb6c6ce7c30729e6e251fd87f043003be9712160b WatchSource:0}: Error finding container e2c63de1aab5663dbeb8089cb6c6ce7c30729e6e251fd87f043003be9712160b: Status 404 returned error can't find the container with id e2c63de1aab5663dbeb8089cb6c6ce7c30729e6e251fd87f043003be9712160b Oct 30 19:30:32.650662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.650622 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:32.650774 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:32.650750 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:33.297461 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.297415 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" event={"ID":"a7b3c348-5b38-4a00-b541-36ecdda3b032","Type":"ContainerStarted","Data":"2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3"} Oct 30 19:30:33.297661 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.297583 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" event={"ID":"a7b3c348-5b38-4a00-b541-36ecdda3b032","Type":"ContainerStarted","Data":"e2c63de1aab5663dbeb8089cb6c6ce7c30729e6e251fd87f043003be9712160b"} Oct 30 19:30:33.299822 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.298604 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:33.300221 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.300198 5627 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-mtqgv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:30:33.300308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.300260 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" podUID="a7b3c348-5b38-4a00-b541-36ecdda3b032" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:30:33.304368 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.301905 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 19:30:33.304368 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.301966 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 19:30:33.561101 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.560963 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:33.789907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:33.789875 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:34.301961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:34.301929 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 19:30:34.302317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:34.302271 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 19:30:34.308841 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:34.308819 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:30:34.357729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:34.357243 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" podStartSLOduration=3.357220816 podStartE2EDuration="3.357220816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:33.334934392 +0000 UTC m=+654.061751675" watchObservedRunningTime="2026-10-30 19:30:34.357220816 +0000 UTC m=+655.084038114" Oct 30 19:30:35.301193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:35.301156 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 19:30:35.301839 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:35.301789 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 19:30:35.720855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:35.720746 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:35.813922 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:35.813857 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:35.853841 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:35.853798 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:36.868189 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:36.868139 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:36.911372 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:36.911346 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:30:36.963021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:36.962991 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:30:37.350904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:37.350854 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:30:37.351124 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:37.350914 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:30:37.625288 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:37.625195 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:37.625646 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:37.625618 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:37.626451 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:37.626436 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:30:37.658258 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:37.658219 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:38.631668 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:38.631608 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:38.764934 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:38.764896 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:39.700010 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:39.699972 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:40.404891 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:30:40.404853 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:30:40.788946 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:40.788911 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:41.717477 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:41.717441 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:42.593401 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:42.593368 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:43.614823 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:43.614789 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:44.541764 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:44.541684 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:45.106834 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:45.106797 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:45.751484 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:45.751453 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:46.831183 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:46.831155 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:47.935424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:47.935396 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:49.048182 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:49.048146 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:49.705754 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:49.705697 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:50.544493 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:50.544463 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:51.558087 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:51.558047 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:52.676641 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:52.676606 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:53.420396 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:53.420367 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:53.578840 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:53.578810 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:54.365835 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:54.365798 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:54.891913 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:54.891857 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:55.457858 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:55.457828 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:56.687728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:56.676413 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:58.023615 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.023581 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k"] Oct 30 19:30:58.027742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.027722 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.039230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.039206 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k"] Oct 30 19:30:58.133466 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133433 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.133616 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133487 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.133616 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133599 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.133691 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133635 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.133760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133740 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.133798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133791 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxmg7\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-kube-api-access-nxmg7\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.133828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133809 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.133865 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133840 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.133899 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.133861 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.224176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.224150 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:58.234691 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.234663 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.234832 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.234726 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.234832 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.234767 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.234832 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.234808 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxmg7\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-kube-api-access-nxmg7\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.234834 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.234876 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.234901 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235278 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.235250 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235373 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.235285 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235445 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.235398 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235503 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.235442 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235503 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.235449 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.235540 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.235669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.235650 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.237156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.237139 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.237961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.237938 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.246151 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.246125 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxmg7\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-kube-api-access-nxmg7\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.247818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.247797 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.342973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.342887 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:30:58.699979 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.699951 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k"] Oct 30 19:30:58.701172 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:30:58.701145 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc6c5c7_8fcb_436d_a86b_4d9f31fd13db.slice/crio-0d97d08704da6060ffd9a2ed84e45f2fffbaa5e1e9b9b82b19d791b7df814e21 WatchSource:0}: Error finding container 0d97d08704da6060ffd9a2ed84e45f2fffbaa5e1e9b9b82b19d791b7df814e21: Status 404 returned error can't find the container with id 0d97d08704da6060ffd9a2ed84e45f2fffbaa5e1e9b9b82b19d791b7df814e21 Oct 30 19:30:58.703766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.703728 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:58.703845 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.703812 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:58.703894 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:58.703879 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:30:59.504201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:59.504157 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" event={"ID":"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db","Type":"ContainerStarted","Data":"261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9"} Oct 30 19:30:59.504608 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:59.504220 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" event={"ID":"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db","Type":"ContainerStarted","Data":"0d97d08704da6060ffd9a2ed84e45f2fffbaa5e1e9b9b82b19d791b7df814e21"} Oct 30 19:30:59.510531 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:30:59.510506 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:00.344427 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:00.344397 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:31:00.344914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:00.344881 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Oct 30 19:31:00.345120 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:00.345089 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Oct 30 19:31:00.401952 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:00.401919 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:00.811888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:00.811855 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:01.344198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:01.344161 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Oct 30 19:31:01.344389 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:01.344216 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Oct 30 19:31:01.743623 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:01.743562 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:02.344231 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:02.344199 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Oct 30 19:31:02.344630 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:02.344253 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Oct 30 19:31:02.839064 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:02.839031 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:03.344186 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:03.344147 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Oct 30 19:31:03.344343 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:03.344208 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Oct 30 19:31:03.760099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:03.760071 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:04.343713 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:04.343658 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Oct 30 19:31:04.343901 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:04.343752 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Oct 30 19:31:04.904842 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:04.904810 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:05.343633 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:05.343601 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Oct 30 19:31:05.343812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:05.343652 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Oct 30 19:31:05.808025 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:05.807985 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:06.347440 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:06.347412 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:31:06.347936 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:06.347918 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:31:06.348514 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:06.348495 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:31:06.374807 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:06.374764 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" podStartSLOduration=8.374751969 podStartE2EDuration="8.374751969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:59.547649784 +0000 UTC m=+680.274467073" watchObservedRunningTime="2026-10-30 19:31:06.374751969 +0000 UTC m=+687.101569266" Oct 30 19:31:06.700691 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:06.700616 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:07.514629 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:07.514589 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:08.222032 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:08.221996 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:09.217097 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:09.217065 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:10.221974 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:10.221927 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:10.403751 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:31:10.403694 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:31:11.015164 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:11.015133 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:11.821550 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:11.821521 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:12.716642 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:12.716598 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:13.078085 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.078046 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4"] Oct 30 19:31:13.084228 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.084202 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.105812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.105791 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4"] Oct 30 19:31:13.152999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.152946 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.153288 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.153271 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.153476 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.153440 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.154014 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.153972 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.154180 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.154032 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.154180 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.154083 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c6b620b-3265-4db3-8789-d510ddc149d7-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.154180 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.154112 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.154180 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.154145 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.154472 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.154252 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnh94\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-kube-api-access-cnh94\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.255924 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.255889 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.255940 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.255974 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c6b620b-3265-4db3-8789-d510ddc149d7-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.255993 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256017 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256106 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnh94\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-kube-api-access-cnh94\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256145 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256179 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256213 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256532 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256347 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256532 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256487 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256683 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256654 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.256810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256778 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c6b620b-3265-4db3-8789-d510ddc149d7-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.257092 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.256845 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.258810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.258783 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.259195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.259176 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.265580 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.265557 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnh94\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-kube-api-access-cnh94\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.266320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.266290 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.433781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.433667 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:13.549732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.549670 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq"] Oct 30 19:31:13.563941 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.563909 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.585261 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.585227 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq"] Oct 30 19:31:13.708010 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.707933 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.708238 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.708214 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdcvx\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-kube-api-access-bdcvx\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.708465 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.708446 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.708897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.708874 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.709100 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.709081 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.709235 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.709217 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4989abce-250d-49ac-9fbb-b45d09a852a2-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.709394 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.709378 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.709527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.709509 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.709902 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.709881 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.789216 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.789141 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.829562 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.829616 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4989abce-250d-49ac-9fbb-b45d09a852a2-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.829679 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.829726 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.829817 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.829926 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.829956 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdcvx\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-kube-api-access-bdcvx\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.830069 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.830749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.830107 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.833924 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.833298 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.833924 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.833594 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.833924 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.833686 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4989abce-250d-49ac-9fbb-b45d09a852a2-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.833924 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.833880 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.834246 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.833960 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.837630 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.837186 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.837630 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.837586 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.844633 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.844604 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdcvx\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-kube-api-access-bdcvx\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.850907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.850882 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:13.908474 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.899878 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4"] Oct 30 19:31:13.919746 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:31:13.914129 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6b620b_3265_4db3_8789_d510ddc149d7.slice/crio-c2445c3f4a4d65b8b4a577f7c67f137e1f5647a505b4e10edce63dfad2b67609 WatchSource:0}: Error finding container c2445c3f4a4d65b8b4a577f7c67f137e1f5647a505b4e10edce63dfad2b67609: Status 404 returned error can't find the container with id c2445c3f4a4d65b8b4a577f7c67f137e1f5647a505b4e10edce63dfad2b67609 Oct 30 19:31:13.919746 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.917332 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:13.919746 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.917421 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:13.919746 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.917468 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:13.955572 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:13.950996 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:14.149612 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.149583 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn"] Oct 30 19:31:14.257783 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.257535 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:14.513186 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.513109 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn"] Oct 30 19:31:14.513575 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.513483 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.535454 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.535415 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"distinct-2-w4-3bc085-kserve-self-signed-certs\"" Oct 30 19:31:14.673257 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.673217 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm4tf\" (UniqueName: \"kubernetes.io/projected/2350a8f2-ee45-47dc-aca4-138b0a907df8-kube-api-access-hm4tf\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.673554 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.673531 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2350a8f2-ee45-47dc-aca4-138b0a907df8-tls-certs\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.673808 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.673786 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-model-cache\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.679910 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.679876 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-tmp-dir\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.680170 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.680146 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-dshm\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.680303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.680287 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-home\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.737696 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.737659 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" event={"ID":"1c6b620b-3265-4db3-8789-d510ddc149d7","Type":"ContainerStarted","Data":"cef44b9b2ea958e9941b52967d54e86a9e71e8a8de7ba26fc12f7dff76fda398"} Oct 30 19:31:14.738008 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.737978 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" event={"ID":"1c6b620b-3265-4db3-8789-d510ddc149d7","Type":"ContainerStarted","Data":"c2445c3f4a4d65b8b4a577f7c67f137e1f5647a505b4e10edce63dfad2b67609"} Oct 30 19:31:14.837783 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.823261 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm4tf\" (UniqueName: \"kubernetes.io/projected/2350a8f2-ee45-47dc-aca4-138b0a907df8-kube-api-access-hm4tf\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.878742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.878548 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2350a8f2-ee45-47dc-aca4-138b0a907df8-tls-certs\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.878742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.878646 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-model-cache\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.879016 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.878844 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-tmp-dir\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.879016 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.878935 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-dshm\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.879016 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.878963 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-home\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.943733 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.943235 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-home\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.953738 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.944241 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-model-cache\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:14.983685 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:14.979071 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-tmp-dir\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:15.007949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.007035 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:15.007949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.007128 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:15.007949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.007200 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:15.023724 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.021350 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podStartSLOduration=2.021330276 podStartE2EDuration="2.021330276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:14.839800048 +0000 UTC m=+695.566617346" watchObservedRunningTime="2026-10-30 19:31:15.021330276 +0000 UTC m=+695.748147574" Oct 30 19:31:15.226774 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.224413 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq"] Oct 30 19:31:15.226774 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.225321 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2350a8f2-ee45-47dc-aca4-138b0a907df8-tls-certs\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:15.336729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.297651 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm4tf\" (UniqueName: \"kubernetes.io/projected/2350a8f2-ee45-47dc-aca4-138b0a907df8-kube-api-access-hm4tf\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:15.341727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.337975 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-dshm\") pod \"distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:15.436821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.436790 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Oct 30 19:31:15.496461 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.496425 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Oct 30 19:31:15.506946 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.506921 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:15.515212 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.511567 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:15.855746 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.851780 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:15.868727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.866751 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" event={"ID":"4989abce-250d-49ac-9fbb-b45d09a852a2","Type":"ContainerStarted","Data":"29926ac4e7376a90c3c773212647b387f293447ffd73b677fcfc1c8fef2cbdb9"} Oct 30 19:31:16.013729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:15.952980 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:16.018950 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:16.018907 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded" Oct 30 19:31:16.437433 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:16.437398 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Oct 30 19:31:16.438257 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:16.438228 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Oct 30 19:31:16.739795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:16.714877 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:31:16.739795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:16.737734 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded" Oct 30 19:31:16.884731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:16.853328 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podStartSLOduration=3.853306651 podStartE2EDuration="3.853306651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:15.932235436 +0000 UTC m=+696.659052735" watchObservedRunningTime="2026-10-30 19:31:16.853306651 +0000 UTC m=+697.580123950" Oct 30 19:31:17.010540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.010452 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn"] Oct 30 19:31:17.010798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:16.986860 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 19:31:17.010942 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.010921 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 19:31:17.011279 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:16.859422 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:31:17.038728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.009312 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:17.039660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.013566 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:31:17.040968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.039379 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:17.041434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.041417 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:31:17.182730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.182006 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:17.301133 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.301041 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:17.312804 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.312773 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:31:17.442082 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.442048 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Oct 30 19:31:17.442628 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.442597 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Oct 30 19:31:17.597212 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.597122 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:31:17.597721 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.597669 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" event={"ID":"4989abce-250d-49ac-9fbb-b45d09a852a2","Type":"ContainerStarted","Data":"4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef"} Oct 30 19:31:17.601632 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.600973 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" event={"ID":"2350a8f2-ee45-47dc-aca4-138b0a907df8","Type":"ContainerStarted","Data":"9bd4728ccac3ceee52828798ae7b9e5ea0ef4e3049aca37702a0a596893558e6"} Oct 30 19:31:17.970986 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.970899 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 19:31:17.971249 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:17.971223 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 19:31:18.166731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:18.163347 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:18.443731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:18.438976 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Oct 30 19:31:18.443731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:18.439036 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Oct 30 19:31:18.608391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:18.608360 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" event={"ID":"2350a8f2-ee45-47dc-aca4-138b0a907df8","Type":"ContainerStarted","Data":"3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3"} Oct 30 19:31:18.679495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:18.679416 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podStartSLOduration=4.679394398 podStartE2EDuration="4.679394398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:18.635069888 +0000 UTC m=+699.361887187" watchObservedRunningTime="2026-10-30 19:31:18.679394398 +0000 UTC m=+699.406211695" Oct 30 19:31:18.971768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:18.954169 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 19:31:18.971768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:18.954228 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 19:31:19.451732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:19.449795 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Oct 30 19:31:19.451732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:19.449853 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Oct 30 19:31:19.513731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:19.512477 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:19.962729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:19.955087 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 19:31:19.962729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:19.955157 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 19:31:20.448734 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:20.437052 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Oct 30 19:31:20.448734 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:20.437113 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Oct 30 19:31:20.732789 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:20.731864 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:20.952730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:20.952045 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 19:31:20.952730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:20.952108 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 19:31:21.440730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:21.437792 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Oct 30 19:31:21.440730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:21.437849 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Oct 30 19:31:21.848586 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:21.848547 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:21.952099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:21.952064 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 19:31:21.952369 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:21.952340 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 19:31:21.999725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:21.998153 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:22.439731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:22.439033 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Oct 30 19:31:22.439731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:22.439094 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Oct 30 19:31:22.952164 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:22.952131 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 19:31:22.952615 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:22.952590 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 19:31:23.228337 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:23.223281 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:23.439548 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:23.439520 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:23.439807 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:23.439779 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:23.440512 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:23.440494 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:31:23.951647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:23.951604 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Oct 30 19:31:23.951647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:23.951646 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Oct 30 19:31:24.658112 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:24.657972 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:24.956515 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:24.956442 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:24.956832 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:24.956814 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:24.957364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:24.957348 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:31:25.513053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:25.513020 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:25.525173 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:25.525146 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:25.741464 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:25.741433 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:25.770516 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:25.770436 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:26.632559 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:26.632528 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:27.727467 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:27.727433 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:28.417084 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.417055 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:28.464757 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.464729 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5"] Oct 30 19:31:28.473328 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.473294 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.494930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.494898 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5"] Oct 30 19:31:28.574771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.574729 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.575144 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.575108 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.575304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.575282 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.575391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.575332 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.575479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.575451 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.575631 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.575610 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfjxz\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-kube-api-access-wfjxz\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.575841 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.575674 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.575841 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.575814 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d47944-f78b-45f4-a00e-094ddb1ce717-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.575996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.575964 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.600818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.598971 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j"] Oct 30 19:31:28.603802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.603775 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.621777 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.620512 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j"] Oct 30 19:31:28.677267 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.677190 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.677424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.677279 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfjxz\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-kube-api-access-wfjxz\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.677424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.677308 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsgb9\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-kube-api-access-dsgb9\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.677424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.677363 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.677424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.677392 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d47944-f78b-45f4-a00e-094ddb1ce717-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.677652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.677630 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.678137 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678115 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d47944-f78b-45f4-a00e-094ddb1ce717-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.678263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678163 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.678263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678234 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.678356 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678323 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.678406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678365 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.678468 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678452 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.678526 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678515 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.678569 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678535 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.678619 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678592 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.678663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678653 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.678809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678783 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.678882 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678856 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.678947 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678919 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.678998 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678947 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.678998 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.678953 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.679236 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.679207 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.679308 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.679256 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.680480 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.680459 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.680937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.680913 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.694837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.694819 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfjxz\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-kube-api-access-wfjxz\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.697258 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.697237 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.720198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.720173 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf"] Oct 30 19:31:28.726145 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.726128 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.737389 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.737365 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf"] Oct 30 19:31:28.779710 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.779681 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.779886 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.779823 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsgb9\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-kube-api-access-dsgb9\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.779886 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.779881 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.780003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.779923 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.780003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.779970 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.780105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.779999 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.780105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.780070 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.780211 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.780155 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.780211 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.780204 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.780328 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.780301 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.780971 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.780948 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.781321 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.781300 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.781675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.781646 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.782060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.782031 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.783062 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.783043 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.785199 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.785174 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.792027 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.792004 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsgb9\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-kube-api-access-dsgb9\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.792168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.792082 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.803009 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.802988 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.884479 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.884567 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.884726 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.884818 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.884857 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.884900 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.884982 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.885019 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.888332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.885056 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqc6t\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-kube-api-access-rqc6t\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.959031 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.958967 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:28.986413 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986382 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.986522 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986442 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqc6t\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-kube-api-access-rqc6t\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.986580 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986537 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.986648 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986628 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.986769 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986749 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.986843 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986800 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.986950 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986928 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.987204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986985 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.987204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.986987 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.987204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.987038 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.987204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.987064 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.987204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.987149 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.987204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.987194 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.987751 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.987731 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.989152 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.989121 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.989463 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.989445 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.995850 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.995819 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqc6t\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-kube-api-access-rqc6t\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:28.995850 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:28.995837 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:29.046513 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.046486 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:29.101998 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.101964 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5"] Oct 30 19:31:29.104090 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:31:29.104058 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d47944_f78b_45f4_a00e_094ddb1ce717.slice/crio-d9aa02b7dc3899c87d99ced86c96b2b384975680aec6eba308d14e3e86544928 WatchSource:0}: Error finding container d9aa02b7dc3899c87d99ced86c96b2b384975680aec6eba308d14e3e86544928: Status 404 returned error can't find the container with id d9aa02b7dc3899c87d99ced86c96b2b384975680aec6eba308d14e3e86544928 Oct 30 19:31:29.107501 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.107456 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.107600 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.107543 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.107600 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.107574 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.244875 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:31:29.244846 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb56101b9_7ef6_4d44_a239_e509c2c49ba3.slice/crio-be910af3ff33ae72aae5a6c8d01a12d0e5b3483cb5e1f99735378984c178452e WatchSource:0}: Error finding container be910af3ff33ae72aae5a6c8d01a12d0e5b3483cb5e1f99735378984c178452e: Status 404 returned error can't find the container with id be910af3ff33ae72aae5a6c8d01a12d0e5b3483cb5e1f99735378984c178452e Oct 30 19:31:29.244997 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.244961 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j"] Oct 30 19:31:29.247585 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.247548 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.247695 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.247630 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.247695 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.247676 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.357746 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.355594 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf"] Oct 30 19:31:29.358011 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.357976 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.358195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.358173 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.358281 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.358230 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:29.504561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.504478 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:29.777304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.776455 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" event={"ID":"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17","Type":"ContainerStarted","Data":"d4348502a8d881932bf1ac17ebcd5e6458912364c576b49ac0d281bf59177315"} Oct 30 19:31:29.777304 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.776520 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" event={"ID":"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17","Type":"ContainerStarted","Data":"f00dd2a04ca829b90d21a2e26769cd8525d30e5e9f9cb133236e753f6221f969"} Oct 30 19:31:29.779208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.778667 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" event={"ID":"b56101b9-7ef6-4d44-a239-e509c2c49ba3","Type":"ContainerStarted","Data":"62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439"} Oct 30 19:31:29.779208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.778735 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" event={"ID":"b56101b9-7ef6-4d44-a239-e509c2c49ba3","Type":"ContainerStarted","Data":"be910af3ff33ae72aae5a6c8d01a12d0e5b3483cb5e1f99735378984c178452e"} Oct 30 19:31:29.780610 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.780585 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" event={"ID":"46d47944-f78b-45f4-a00e-094ddb1ce717","Type":"ContainerStarted","Data":"72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77"} Oct 30 19:31:29.780804 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.780783 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" event={"ID":"46d47944-f78b-45f4-a00e-094ddb1ce717","Type":"ContainerStarted","Data":"d9aa02b7dc3899c87d99ced86c96b2b384975680aec6eba308d14e3e86544928"} Oct 30 19:31:29.802885 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.802832 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podStartSLOduration=1.802816459 podStartE2EDuration="1.802816459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:29.800594454 +0000 UTC m=+710.527411752" watchObservedRunningTime="2026-10-30 19:31:29.802816459 +0000 UTC m=+710.529633756" Oct 30 19:31:29.803138 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.803120 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:29.847736 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.847141 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podStartSLOduration=1.847115809 podStartE2EDuration="1.847115809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:29.846341067 +0000 UTC m=+710.573158370" watchObservedRunningTime="2026-10-30 19:31:29.847115809 +0000 UTC m=+710.573933108" Oct 30 19:31:29.896833 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.896305 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" podStartSLOduration=1.896283463 podStartE2EDuration="1.896283463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:29.879359047 +0000 UTC m=+710.606176345" watchObservedRunningTime="2026-10-30 19:31:29.896283463 +0000 UTC m=+710.623100763" Oct 30 19:31:29.959137 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:29.959106 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:30.058800 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:30.050780 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:30.058800 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:30.050904 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 19:31:30.058800 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:30.050952 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 19:31:30.566181 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:30.566146 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:30.812627 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:30.812595 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Oct 30 19:31:30.813211 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:30.813185 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Oct 30 19:31:30.972739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:30.972618 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 19:31:30.973013 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:30.972983 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 19:31:31.055728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:31.047124 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 19:31:31.055728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:31.047189 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 19:31:31.605891 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:31.605847 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:31.806137 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:31.806102 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Oct 30 19:31:31.806377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:31.806351 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Oct 30 19:31:31.989790 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:31.978970 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 19:31:31.989790 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:31.979045 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 19:31:32.050138 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:32.050102 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 19:31:32.050385 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:32.050358 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 19:31:32.803195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:32.803153 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:32.803889 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:32.803863 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Oct 30 19:31:32.804135 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:32.804082 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Oct 30 19:31:32.983632 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:32.983595 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 19:31:33.017860 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:33.017822 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 19:31:33.047875 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:33.047845 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 19:31:33.048182 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:33.048139 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 19:31:33.803602 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:33.803556 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Oct 30 19:31:33.803911 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:33.803872 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Oct 30 19:31:33.894725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:33.894592 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:33.968725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:33.961079 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 19:31:33.968725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:33.961140 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 19:31:34.049743 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:34.049022 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 19:31:34.049743 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:34.049092 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 19:31:34.803623 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:34.803596 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Oct 30 19:31:34.803860 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:34.803838 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Oct 30 19:31:34.903828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:34.903782 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:34.964730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:34.963084 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 19:31:34.964730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:34.963150 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 19:31:35.064639 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:35.064542 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 19:31:35.065286 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:35.065254 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 19:31:35.804144 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:35.804106 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Oct 30 19:31:35.804411 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:35.804383 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Oct 30 19:31:35.964087 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:35.964045 5627 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Oct 30 19:31:35.964342 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:35.964313 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Oct 30 19:31:36.012350 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:36.012298 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:36.064733 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:36.064636 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 19:31:36.065277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:36.065244 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 19:31:36.803893 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:36.803854 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Oct 30 19:31:36.804112 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:36.803921 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Oct 30 19:31:37.029837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.029783 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:37.363201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.363170 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:37.364474 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.364336 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:37.368791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.365334 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:31:37.800780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.800750 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:37.801147 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.801129 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:37.801963 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.801944 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:31:37.809607 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.809583 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:37.810033 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.810006 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:37.810815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:37.810793 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:31:38.028905 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:38.028869 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:38.924598 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:38.924567 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:39.792455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:39.792420 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:40.404945 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:31:40.404895 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:31:40.700961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:40.700879 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:41.593566 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:41.593533 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:42.490595 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:42.490562 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:43.304228 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.304188 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv"] Oct 30 19:31:43.310880 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.310857 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.328386 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.328361 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv"] Oct 30 19:31:43.426067 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426034 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.426245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426097 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.426245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426198 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.426361 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426254 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/093f3b61-0d77-473c-be63-9b96facf3b61-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.426361 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426341 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.426461 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426379 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/093f3b61-0d77-473c-be63-9b96facf3b61-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.426461 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426413 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6zm5\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-kube-api-access-f6zm5\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.426569 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426536 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.426633 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.426617 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.529629 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.529788 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.529883 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.529978 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.530035 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/093f3b61-0d77-473c-be63-9b96facf3b61-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.530124 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.530193 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/093f3b61-0d77-473c-be63-9b96facf3b61-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.530239 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6zm5\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-kube-api-access-f6zm5\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.531727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.530294 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.548221 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.548150 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:43.550325 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.550298 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.554567 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.551414 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.555215 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.555188 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.556305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.556284 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.557093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.557035 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/093f3b61-0d77-473c-be63-9b96facf3b61-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.567295 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.567268 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.569491 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.569459 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.569681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.569653 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/093f3b61-0d77-473c-be63-9b96facf3b61-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.569681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.569784 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6zm5\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-kube-api-access-f6zm5\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.627812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.627771 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:43.931521 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.931488 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv"] Oct 30 19:31:43.932070 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:31:43.932041 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093f3b61_0d77_473c_be63_9b96facf3b61.slice/crio-f1eb1ed389dcaa079e105dc78cd6278a90ee4b0576f78dd0868bc7344cbd8013 WatchSource:0}: Error finding container f1eb1ed389dcaa079e105dc78cd6278a90ee4b0576f78dd0868bc7344cbd8013: Status 404 returned error can't find the container with id f1eb1ed389dcaa079e105dc78cd6278a90ee4b0576f78dd0868bc7344cbd8013 Oct 30 19:31:43.935213 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.935177 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:43.935332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.935242 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:43.935332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:43.935271 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:44.203512 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:44.203423 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" event={"ID":"093f3b61-0d77-473c-be63-9b96facf3b61","Type":"ContainerStarted","Data":"b4a355e5d99834716773729fe8d02336f1c98baf0677b8ecb44f2ac896d23dff"} Oct 30 19:31:44.203512 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:44.203501 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" event={"ID":"093f3b61-0d77-473c-be63-9b96facf3b61","Type":"ContainerStarted","Data":"f1eb1ed389dcaa079e105dc78cd6278a90ee4b0576f78dd0868bc7344cbd8013"} Oct 30 19:31:44.234293 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:44.234246 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podStartSLOduration=1.234232283 podStartE2EDuration="1.234232283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:44.232228518 +0000 UTC m=+724.959045814" watchObservedRunningTime="2026-10-30 19:31:44.234232283 +0000 UTC m=+724.961049580" Oct 30 19:31:44.532220 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:44.532166 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:44.630867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:44.630834 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:45.356121 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:45.356058 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:45.641100 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:45.641009 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:45.641722 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:45.641667 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:45.682838 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:45.682792 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:45.794732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:45.783667 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:46.007907 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.007850 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded" Oct 30 19:31:46.095880 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.095836 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": context deadline exceeded" Oct 30 19:31:46.329814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.328106 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:46.433746 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.432815 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:46.433746 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.433588 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:46.523753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.523545 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:46.630246 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.630151 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:46.630508 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.630478 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:46.694891 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.694831 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:46.696921 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.696847 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:31:46.697301 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.697284 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:31:46.800834 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.800793 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:46.806426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:46.806401 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:47.041727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.017026 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:47.212597 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.170038 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:31:47.212597 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.103960 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:47.212597 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.170793 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:31:47.381896 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.360181 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:47.579729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.402829 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:31:47.579729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.473196 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded" Oct 30 19:31:47.582088 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.473549 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:47.633838 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.633753 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:47.634072 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.634045 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:47.696973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.696932 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:47.720814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.718954 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:31:47.833686 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.833635 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:47.928810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.910066 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:31:47.928810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.910137 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:31:47.928810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.910172 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:31:47.928810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.910204 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:31:47.928810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:47.913000 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:48.177964 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.177924 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded" Oct 30 19:31:48.182859 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.182747 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:48.607147 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.605958 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded" Oct 30 19:31:48.607147 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.606341 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:31:48.607147 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.606621 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded" Oct 30 19:31:48.628762 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.628666 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:48.629881 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.629018 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:48.702016 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.701975 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded" Oct 30 19:31:48.728848 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.728802 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:31:48.866786 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:48.846652 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded" Oct 30 19:31:49.205275 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.205176 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:49.207264 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.207233 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:49.609153 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.609104 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:31:49.609620 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.609415 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:49.612065 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.609055 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:49.631195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.631164 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:49.631466 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.631424 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:49.703869 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.703791 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:49.729232 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.729194 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:49.943234 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:49.941226 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded" Oct 30 19:31:50.090894 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.090848 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:50.223140 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.222988 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded" Oct 30 19:31:50.264766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.263802 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": context deadline exceeded" Oct 30 19:31:50.325828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.315671 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:31:50.658776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.646183 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded" Oct 30 19:31:50.713797 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.650603 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:50.716184 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.703995 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:31:50.716853 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.713005 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:50.731025 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.730819 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:50.732146 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.732116 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:31:50.819738 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.819676 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:50.964313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:50.964214 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:51.232791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.224153 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:51.264860 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.264798 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:31:51.380035 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.380006 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:31:51.392975 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.392938 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:31:51.546730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.538684 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:31:51.547867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.547444 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:31:51.550169 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.549843 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:31:51.677984 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.677884 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:31:51.695973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.695944 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:51.696219 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.696191 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:51.697278 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:51.697249 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:31:52.640730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:52.639985 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:52.640730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:52.640041 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:53.641244 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:53.641192 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:53.641810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:53.641257 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:53.774011 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:53.773976 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:54.635313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:54.635279 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:54.635603 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:54.635563 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:54.858050 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:54.858018 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:55.628655 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:55.628622 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Oct 30 19:31:55.628841 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:55.628684 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Oct 30 19:31:55.874944 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:55.874892 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:56.579494 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:56.579465 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:57.293595 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:57.293564 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:57.294250 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:57.294195 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:57.295088 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:57.295063 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:31:57.560043 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:57.559953 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:58.167407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.167368 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb"] Oct 30 19:31:58.171019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.171000 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.200958 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.200935 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb"] Oct 30 19:31:58.255348 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.255315 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.255496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.255466 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.255577 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.255495 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf02d4a-73a3-4e29-9504-64c516a4869a-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.255777 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.255749 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.255887 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.255809 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.255953 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.255914 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.255953 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.255948 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.256071 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.256056 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.256126 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.256112 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48h6v\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-kube-api-access-48h6v\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.363564 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.363653 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.363742 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.363779 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48h6v\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-kube-api-access-48h6v\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.363827 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.363933 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.363972 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf02d4a-73a3-4e29-9504-64c516a4869a-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.364124 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.364203 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.365669 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.366735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.366305 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.369159 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.367398 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf02d4a-73a3-4e29-9504-64c516a4869a-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.369159 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.367746 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.369440 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.369357 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:58.369440 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.369420 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.377231 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.377165 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.377607 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.377580 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.379370 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.379343 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.380079 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.380015 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48h6v\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-kube-api-access-48h6v\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.500218 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.500183 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:58.795927 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.795898 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb"] Oct 30 19:31:58.796929 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:31:58.796904 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf02d4a_73a3_4e29_9504_64c516a4869a.slice/crio-7799ab56cd23d5310e5e0f2b847800602909139b7d5cbcac25e6a5caf273c343 WatchSource:0}: Error finding container 7799ab56cd23d5310e5e0f2b847800602909139b7d5cbcac25e6a5caf273c343: Status 404 returned error can't find the container with id 7799ab56cd23d5310e5e0f2b847800602909139b7d5cbcac25e6a5caf273c343 Oct 30 19:31:58.799323 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.799293 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:58.799417 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.799359 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:58.799417 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:58.799399 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:31:59.034111 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:59.034073 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" event={"ID":"ebf02d4a-73a3-4e29-9504-64c516a4869a","Type":"ContainerStarted","Data":"4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd"} Oct 30 19:31:59.034297 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:59.034148 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" event={"ID":"ebf02d4a-73a3-4e29-9504-64c516a4869a","Type":"ContainerStarted","Data":"7799ab56cd23d5310e5e0f2b847800602909139b7d5cbcac25e6a5caf273c343"} Oct 30 19:31:59.062863 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:59.062749 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podStartSLOduration=1.0627292640000001 podStartE2EDuration="1.062729264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:59.058682392 +0000 UTC m=+739.785499688" watchObservedRunningTime="2026-10-30 19:31:59.062729264 +0000 UTC m=+739.789546560" Oct 30 19:31:59.313053 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:59.312953 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:59.501411 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:59.501365 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:31:59.503049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:59.503025 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 19:31:59.503154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:31:59.503075 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 19:32:00.241783 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:00.241747 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:00.500739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:00.500639 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 19:32:00.500905 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:00.500752 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 19:32:01.047254 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:01.047217 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:01.254064 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:01.254033 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:01.506462 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:01.506425 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 19:32:01.506669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:01.506488 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 19:32:02.441605 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:02.441535 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:02.501211 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:02.501181 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 19:32:02.501391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:02.501234 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 19:32:03.512168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:03.511398 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 19:32:03.512168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:03.511467 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 19:32:03.652723 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:03.649147 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:04.500810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:04.500772 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 19:32:04.500981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:04.500840 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 19:32:04.669591 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:04.669558 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:05.501461 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:05.501432 5627 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Oct 30 19:32:05.501630 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:05.501493 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Oct 30 19:32:05.801177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:05.801088 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:06.197207 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:06.197128 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn"] Oct 30 19:32:06.323576 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:06.323537 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:06.593282 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:06.593243 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:32:06.593696 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:06.593674 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:32:06.594372 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:06.594351 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:32:06.712650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:06.712616 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:07.671141 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:07.671106 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:09.612766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:09.607948 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:09.614933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:09.613567 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:09.614933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:09.613899 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded" Oct 30 19:32:09.717636 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:09.714671 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:09.726741 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:09.726433 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:09.949781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:09.945279 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:10.108731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.094079 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:10.191728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.191553 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.774179 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.775397 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.775653 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.775897 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.776788 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.776847 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.776885 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.776918 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.779575 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded" Oct 30 19:32:10.794818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.779910 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:32:10.828746 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.818933 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": context deadline exceeded" Oct 30 19:32:10.888510 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:32:10.888466 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:32:10.915179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.893603 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:10.968328 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:10.945876 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded" Oct 30 19:32:11.234846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.195081 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:32:11.234846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.196928 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:32:11.234846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.196992 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:11.234846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.197847 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:11.234846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.219150 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:32:11.260733 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.249321 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:11.602854 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.273280 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:11.783081 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.783032 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:11.785478 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.785444 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:11.786798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.786772 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:11.787984 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.787955 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:11.789126 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:11.789103 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:32:12.044212 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:12.044176 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:32:12.044839 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:12.044807 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:12.196728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:12.193547 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:12.203841 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:12.203802 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:12.616867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:12.616820 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": context deadline exceeded" Oct 30 19:32:12.617988 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:12.617963 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:32:12.660868 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:12.641063 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:32:13.087348 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.087287 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded" Oct 30 19:32:13.262852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.087912 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded" Oct 30 19:32:13.262852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.087953 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:32:13.262852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.087976 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded" Oct 30 19:32:13.262852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.181503 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:13.262852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.190996 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:32:13.262852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.194019 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:13.262852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.230019 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/health\": context deadline exceeded" Oct 30 19:32:13.269006 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.247545 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded" Oct 30 19:32:13.378123 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.378028 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:32:13.379129 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.379105 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:13.415106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.415083 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:32:13.850663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.850635 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:32:13.949005 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.948255 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:32:13.962772 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.962746 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:32:13.963773 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:13.963753 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:32:15.971048 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:15.971009 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:16.479336 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:16.479305 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:17.325150 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:17.325088 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:18.117836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:18.117807 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:18.926401 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:18.926362 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:20.012550 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:20.012520 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:20.576763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:20.576731 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:20.988130 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:20.988046 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:21.228212 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.228172 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" containerID="cri-o://ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9" gracePeriod=45 Oct 30 19:32:21.523852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.523501 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:32:21.601995 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.601963 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:21.702141 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702107 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tmp-dir\") pod \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " Oct 30 19:32:21.702325 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702158 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-dshm\") pod \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " Oct 30 19:32:21.702393 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702371 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-kube-api-access-mxszc\" (UniqueName: \"kubernetes.io/projected/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-kube-api-access-mxszc\") pod \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " Oct 30 19:32:21.702453 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702400 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tmp-dir" pod "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" (UID: "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:21.702453 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702408 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-home\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-home\") pod \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " Oct 30 19:32:21.702626 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702487 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-model-cache\") pod \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " Oct 30 19:32:21.702626 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702616 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tls-certs\" (UniqueName: \"kubernetes.io/secret/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tls-certs\") pod \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\" (UID: \"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5\") " Oct 30 19:32:21.702787 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702642 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-home" pod "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" (UID: "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:21.702830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.702798 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-model-cache" pod "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" (UID: "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:21.703326 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.703257 5627 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-home\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:21.703326 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.703278 5627 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-model-cache\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:21.703326 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.703286 5627 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tmp-dir\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:21.704615 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.704586 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-dshm" pod "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" (UID: "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:21.704755 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.704726 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-kube-api-access-mxszc" pod "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" (UID: "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5"). InnerVolumeSpecName "kube-api-access-mxszc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:21.704839 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.704821 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tls-certs" pod "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" (UID: "534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:32:21.762885 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.762837 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:21.798798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.796083 5627 generic.go:356] "Generic (PLEG): container finished" podID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerID="ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9" exitCode=0 Oct 30 19:32:21.798798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.796323 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" event={"ID":"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5","Type":"ContainerDied","Data":"ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9"} Oct 30 19:32:21.798798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.796361 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" event={"ID":"534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5","Type":"ContainerDied","Data":"dd2d3b85074f81aeb419c55d16a5767696b76a783c5a648c8701041d2d1d9bc4"} Oct 30 19:32:21.798798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.796387 5627 scope.go:122] "RemoveContainer" containerID="ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9" Oct 30 19:32:21.798798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.796574 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn" Oct 30 19:32:21.822739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.821899 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxszc\" (UniqueName: \"kubernetes.io/projected/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-kube-api-access-mxszc\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:21.822739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.821936 5627 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-tls-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:21.822739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.821953 5627 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5-dshm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:21.966178 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.966076 5627 scope.go:122] "RemoveContainer" containerID="ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9" Oct 30 19:32:21.980920 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:32:21.980865 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9\": container with ID starting with ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9 not found: ID does not exist" containerID="ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9" Oct 30 19:32:21.981096 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.980995 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9"} err="failed to get container status \"ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9\": rpc error: code = NotFound desc = could not find container \"ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9\": container with ID starting with ccddc6e2d7088316aed9e168ff41136ce7be28237c5b51f4038db8793a4062b9 not found: ID does not exist" Oct 30 19:32:21.983359 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.983324 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn"] Oct 30 19:32:21.989817 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:21.989798 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-w3-9f2d57-kserve-5c6bc5d85f-mxwvn"] Oct 30 19:32:22.524275 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:22.524011 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:23.399722 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:23.399671 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" path="/var/lib/kubelet/pods/534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5/volumes" Oct 30 19:32:23.564958 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:23.564926 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:23.695015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:23.694925 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:24.769593 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:24.769561 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:25.812268 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:25.812169 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:26.602297 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:26.602247 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:27.333039 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:27.333004 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:28.278027 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.274923 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:28.471588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.471553 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6"] Oct 30 19:32:28.472447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.472422 5627 cpu_manager.go:414] "containerMap: removing container" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" Oct 30 19:32:28.472447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.472448 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" Oct 30 19:32:28.472660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.472640 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="534e4ef6-7f4f-46db-bf68-8ea9cbaf48f5" containerName="main" Oct 30 19:32:28.476945 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.476924 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.493687 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.493665 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6"] Oct 30 19:32:28.507717 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.507674 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.507834 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.507814 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.507885 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.507846 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.507919 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.507881 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.507961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.507914 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.508051 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.508036 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh5f9\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-kube-api-access-wh5f9\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.508545 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.508521 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.508656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.508599 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.508656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.508636 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.609979 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.609898 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.609979 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.609949 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.609979 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.609970 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610016 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610064 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610082 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610108 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610135 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610263 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610229 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh5f9\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-kube-api-access-wh5f9\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610327 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610392 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610474 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610792 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.610888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.610825 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.612492 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.612471 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.612830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.612812 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.620165 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.620141 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh5f9\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-kube-api-access-wh5f9\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.620267 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.620175 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:28.677608 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.677584 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:28.795179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:28.795133 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:29.091228 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:32:29.091198 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e5a633_4a42_4a8d_9eae_c7805c2cdab2.slice/crio-41b8b37ef30b9f9a2a41ef81dc383eead99a78d260f128d706114ef80a32b31b WatchSource:0}: Error finding container 41b8b37ef30b9f9a2a41ef81dc383eead99a78d260f128d706114ef80a32b31b: Status 404 returned error can't find the container with id 41b8b37ef30b9f9a2a41ef81dc383eead99a78d260f128d706114ef80a32b31b Oct 30 19:32:29.092795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:29.092728 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6"] Oct 30 19:32:29.093902 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:29.093868 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:32:29.093987 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:29.093944 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:32:29.094077 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:29.094062 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:32:29.227989 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:29.227951 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:29.891334 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:29.891294 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" event={"ID":"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2","Type":"ContainerStarted","Data":"c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea"} Oct 30 19:32:29.891793 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:29.891405 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" event={"ID":"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2","Type":"ContainerStarted","Data":"41b8b37ef30b9f9a2a41ef81dc383eead99a78d260f128d706114ef80a32b31b"} Oct 30 19:32:29.943991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:29.943927 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" podStartSLOduration=1.9439064780000002 podStartE2EDuration="1.943906478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:29.931989737 +0000 UTC m=+770.658807038" watchObservedRunningTime="2026-10-30 19:32:29.943906478 +0000 UTC m=+770.670723776" Oct 30 19:32:30.221777 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:30.220217 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:30.795637 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:30.795597 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:30.796981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:30.796955 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:32:30.797116 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:30.797006 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 19:32:31.272288 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:31.272262 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:31.795442 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:31.795408 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:32:31.795607 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:31.795471 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 19:32:32.351179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:32.351145 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:32.544245 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:32.544213 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:32.797277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:32.797244 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:32:32.797565 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:32.797535 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 19:32:33.796154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:33.796121 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:32:33.796506 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:33.796192 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 19:32:34.007570 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.007538 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:34.141380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.141273 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:34.785029 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.784994 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm"] Oct 30 19:32:34.785318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.785216 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" podUID="d18b7b86-e8ee-4b9f-a695-92080e70f540" containerName="maas-api" containerID="cri-o://85786bcffe0acddac0d7e30b82990b5de2a4031758365f91fa698d9cce507a03" gracePeriod=30 Oct 30 19:32:34.796198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.796051 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:32:34.796198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.796092 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 19:32:34.872217 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.872187 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:34.956759 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.956718 5627 generic.go:356] "Generic (PLEG): container finished" podID="d18b7b86-e8ee-4b9f-a695-92080e70f540" containerID="85786bcffe0acddac0d7e30b82990b5de2a4031758365f91fa698d9cce507a03" exitCode=0 Oct 30 19:32:34.956915 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:34.956754 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" event={"ID":"d18b7b86-e8ee-4b9f-a695-92080e70f540","Type":"ContainerDied","Data":"85786bcffe0acddac0d7e30b82990b5de2a4031758365f91fa698d9cce507a03"} Oct 30 19:32:35.231727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.230913 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:32:35.376802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.374733 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d18b7b86-e8ee-4b9f-a695-92080e70f540-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7b86-e8ee-4b9f-a695-92080e70f540-openshift-service-ca\") pod \"d18b7b86-e8ee-4b9f-a695-92080e70f540\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " Oct 30 19:32:35.376802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.374973 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d18b7b86-e8ee-4b9f-a695-92080e70f540-kube-api-access-wrbrt\" (UniqueName: \"kubernetes.io/projected/d18b7b86-e8ee-4b9f-a695-92080e70f540-kube-api-access-wrbrt\") pod \"d18b7b86-e8ee-4b9f-a695-92080e70f540\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " Oct 30 19:32:35.376802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.375036 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-maas-api-tls\") pod \"d18b7b86-e8ee-4b9f-a695-92080e70f540\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " Oct 30 19:32:35.376802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.375096 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-metrics-certs\") pod \"d18b7b86-e8ee-4b9f-a695-92080e70f540\" (UID: \"d18b7b86-e8ee-4b9f-a695-92080e70f540\") " Oct 30 19:32:35.379735 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.378685 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d18b7b86-e8ee-4b9f-a695-92080e70f540-openshift-service-ca" pod "d18b7b86-e8ee-4b9f-a695-92080e70f540" (UID: "d18b7b86-e8ee-4b9f-a695-92080e70f540"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:32:35.386479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.381059 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-maas-api-tls" pod "d18b7b86-e8ee-4b9f-a695-92080e70f540" (UID: "d18b7b86-e8ee-4b9f-a695-92080e70f540"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:32:35.389547 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.389497 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-metrics-certs" pod "d18b7b86-e8ee-4b9f-a695-92080e70f540" (UID: "d18b7b86-e8ee-4b9f-a695-92080e70f540"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:32:35.397884 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.397328 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18b7b86-e8ee-4b9f-a695-92080e70f540-kube-api-access-wrbrt" pod "d18b7b86-e8ee-4b9f-a695-92080e70f540" (UID: "d18b7b86-e8ee-4b9f-a695-92080e70f540"). InnerVolumeSpecName "kube-api-access-wrbrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:35.472124 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.472090 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:35.477610 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.477585 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:35.477610 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.477610 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7b86-e8ee-4b9f-a695-92080e70f540-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:35.477787 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.477619 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrbrt\" (UniqueName: \"kubernetes.io/projected/d18b7b86-e8ee-4b9f-a695-92080e70f540-kube-api-access-wrbrt\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:35.477787 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.477628 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d18b7b86-e8ee-4b9f-a695-92080e70f540-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:35.660224 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.660149 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:35.788579 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.788545 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:35.795652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.795622 5627 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:32:35.795932 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.795908 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 19:32:35.965226 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.965149 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" Oct 30 19:32:35.965226 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.965175 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm" event={"ID":"d18b7b86-e8ee-4b9f-a695-92080e70f540","Type":"ContainerDied","Data":"1a380b371383f0e40a25275053951a841673c40df20f2d33d544ff918153a855"} Oct 30 19:32:35.965776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.965257 5627 scope.go:122] "RemoveContainer" containerID="85786bcffe0acddac0d7e30b82990b5de2a4031758365f91fa698d9cce507a03" Oct 30 19:32:35.988022 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.987995 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm"] Oct 30 19:32:35.992414 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:35.992392 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-d55178-666654c97-g82tm"] Oct 30 19:32:36.181408 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:36.181366 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:36.731404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:36.731369 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:37.031579 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:37.031550 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:37.032208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:37.032181 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:37.033116 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:37.033095 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:32:37.284566 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:37.284487 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb"] Oct 30 19:32:37.410063 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:37.409975 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d18b7b86-e8ee-4b9f-a695-92080e70f540" path="/var/lib/kubelet/pods/d18b7b86-e8ee-4b9f-a695-92080e70f540/volumes" Oct 30 19:32:37.430724 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:37.421394 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:37.826609 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:37.826572 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:39.000287 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:38.999874 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:39.264836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:39.262814 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:39.264836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:39.263161 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded" Oct 30 19:32:39.428650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:39.428311 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:39.793769 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:39.792851 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:39.808221 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:39.808180 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:39.817043 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:39.815843 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:39.980377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:39.980252 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded" Oct 30 19:32:39.993803 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:39.987688 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:40.129732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.126270 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:40.225742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.222817 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:40.266739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.265786 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:40.266739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.266176 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:32:40.267032 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.266794 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:32:40.267032 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.266848 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:32:40.565177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.565072 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:40.566243 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.566222 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:32:40.726730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.718487 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:40.788135 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.787797 5627 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:32:40.788135 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.787856 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:40.815786 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.810111 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded" Oct 30 19:32:40.815786 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.810531 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:32:40.816146 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.816069 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:40.926768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.829915 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:40.926768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:40.831065 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:32:40.985768 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:32:40.974184 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:32:41.103794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.087806 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:41.200941 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.200873 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:32:41.247985 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.247940 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:41.285863 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.285810 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:41.309729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.286769 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:32:41.414818 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.394616 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:32:41.703785 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.324806 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:41.900726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.872468 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:41.987799 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.901080 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded" Oct 30 19:32:41.993739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:41.988112 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj" podUID="8282cab2-7b52-4403-ac9e-a9897438dfba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": context deadline exceeded" Oct 30 19:32:42.064727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.039851 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:32:42.091960 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.091913 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:42.115749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.107671 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bj75q" podUID="2d4749bf-c669-4371-af0f-e675731c3fc2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.29:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:42.117590 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.064995 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:42.159734 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.135844 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:42.190935 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.190897 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-lvm6p" podUID="5a8648c2-b866-44c0-8371-e9223a8741eb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:42.289682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.289630 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:42.317873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.317841 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:42.331573 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.331534 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:32:42.372730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.371822 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:42.994732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.994361 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded" Oct 30 19:32:43.141730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:42.995740 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:43.141730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:43.132060 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded" Oct 30 19:32:43.190732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:43.178337 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:32:43.237728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:43.237010 5627 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/health\": context deadline exceeded" Oct 30 19:32:43.261013 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:43.260880 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:43.341675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:43.317757 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:32:43.400857 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:43.400803 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:43.659816 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:43.642570 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:32:44.030729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.022259 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded" Oct 30 19:32:44.030729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.022621 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:44.356920 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.356815 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:32:44.365734 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.359321 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" podUID="7eb64bc3-1b22-4944-b693-be0833371aaf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Oct 30 19:32:44.377729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.367018 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded" Oct 30 19:32:44.465730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.462635 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded" Oct 30 19:32:44.521732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.519395 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:44.617881 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.617751 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:44.776725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.761063 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:44.970404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:44.966369 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:32:45.022459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.022422 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:45.024193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.024161 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:45.027561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.026201 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:45.164386 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.164358 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:32:45.247922 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.247886 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:45.311478 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.305476 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j"] Oct 30 19:32:45.314765 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.313178 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" containerID="cri-o://62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439" gracePeriod=30 Oct 30 19:32:45.317034 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.317005 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd"] Oct 30 19:32:45.317777 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.317747 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" containerID="cri-o://363497b64f8822ae25ab63dca0365c985be2a0960d75df6abaa5b02bb648b6c4" gracePeriod=30 Oct 30 19:32:45.318661 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.318641 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm" Oct 30 19:32:45.371120 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.371062 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:45.466493 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.466448 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:45.476641 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.476617 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:32:45.476930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.476913 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:32:45.478972 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.478954 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:32:45.519921 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.519830 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:32:45.581080 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.581036 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:32:45.887776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:45.878566 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:46.089372 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:46.089330 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:46.335041 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:46.335010 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:47.984021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:47.983988 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:48.111210 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:48.111182 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:48.230345 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:48.230313 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:48.923824 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:48.923792 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp"] Oct 30 19:32:48.924081 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:48.924057 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" podUID="512cbb25-1c9d-4419-a15c-d954feb58d31" containerName="maas-api" containerID="cri-o://2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241" gracePeriod=30 Oct 30 19:32:49.065546 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.065504 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:49.200407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.200331 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:49.509937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.509902 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:49.541439 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.536546 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf"] Oct 30 19:32:49.541439 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.536614 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg"] Oct 30 19:32:49.541439 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.536842 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" containerID="cri-o://f8a6670d890202195ef26b53d96e6304d480239b8402ff47ba11c1446b02c869" gracePeriod=30 Oct 30 19:32:49.541439 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.537067 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" containerID="cri-o://d4348502a8d881932bf1ac17ebcd5e6458912364c576b49ac0d281bf59177315" gracePeriod=30 Oct 30 19:32:49.597798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.589240 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:32:49.597798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.590844 5627 generic.go:356] "Generic (PLEG): container finished" podID="512cbb25-1c9d-4419-a15c-d954feb58d31" containerID="2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241" exitCode=0 Oct 30 19:32:49.597798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.590933 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" event={"ID":"512cbb25-1c9d-4419-a15c-d954feb58d31","Type":"ContainerDied","Data":"2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241"} Oct 30 19:32:49.597798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.591016 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" event={"ID":"512cbb25-1c9d-4419-a15c-d954feb58d31","Type":"ContainerDied","Data":"be7bca5a7b9af22ab5ba19df1e03c5c3b1ba8c9ba07049b8e3f1845579f861bc"} Oct 30 19:32:49.597798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.591045 5627 scope.go:122] "RemoveContainer" containerID="2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241" Oct 30 19:32:49.635099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.635072 5627 scope.go:122] "RemoveContainer" containerID="2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241" Oct 30 19:32:49.641288 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:32:49.641254 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241\": container with ID starting with 2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241 not found: ID does not exist" containerID="2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241" Oct 30 19:32:49.641528 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.641490 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241"} err="failed to get container status \"2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241\": rpc error: code = NotFound desc = could not find container \"2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241\": container with ID starting with 2161f8beaf78f291bbea16d250f5dcc32828d1faea5a6cb719a22abd88644241 not found: ID does not exist" Oct 30 19:32:49.739732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.738725 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-maas-api-tls\") pod \"512cbb25-1c9d-4419-a15c-d954feb58d31\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " Oct 30 19:32:49.739732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.738773 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/512cbb25-1c9d-4419-a15c-d954feb58d31-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512cbb25-1c9d-4419-a15c-d954feb58d31-openshift-service-ca\") pod \"512cbb25-1c9d-4419-a15c-d954feb58d31\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " Oct 30 19:32:49.739732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.738904 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/512cbb25-1c9d-4419-a15c-d954feb58d31-kube-api-access-msrt5\" (UniqueName: \"kubernetes.io/projected/512cbb25-1c9d-4419-a15c-d954feb58d31-kube-api-access-msrt5\") pod \"512cbb25-1c9d-4419-a15c-d954feb58d31\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " Oct 30 19:32:49.739732 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.739080 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-metrics-certs\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-metrics-certs\") pod \"512cbb25-1c9d-4419-a15c-d954feb58d31\" (UID: \"512cbb25-1c9d-4419-a15c-d954feb58d31\") " Oct 30 19:32:49.741010 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.740981 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/512cbb25-1c9d-4419-a15c-d954feb58d31-openshift-service-ca" pod "512cbb25-1c9d-4419-a15c-d954feb58d31" (UID: "512cbb25-1c9d-4419-a15c-d954feb58d31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:32:49.769796 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.761010 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-metrics-certs" pod "512cbb25-1c9d-4419-a15c-d954feb58d31" (UID: "512cbb25-1c9d-4419-a15c-d954feb58d31"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:32:49.769796 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.764544 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/512cbb25-1c9d-4419-a15c-d954feb58d31-kube-api-access-msrt5" pod "512cbb25-1c9d-4419-a15c-d954feb58d31" (UID: "512cbb25-1c9d-4419-a15c-d954feb58d31"). InnerVolumeSpecName "kube-api-access-msrt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:49.770395 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.770373 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-maas-api-tls" pod "512cbb25-1c9d-4419-a15c-d954feb58d31" (UID: "512cbb25-1c9d-4419-a15c-d954feb58d31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:32:49.841279 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.841238 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msrt5\" (UniqueName: \"kubernetes.io/projected/512cbb25-1c9d-4419-a15c-d954feb58d31-kube-api-access-msrt5\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:49.841279 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.841271 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:49.841279 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.841287 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512cbb25-1c9d-4419-a15c-d954feb58d31-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:49.841610 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.841301 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512cbb25-1c9d-4419-a15c-d954feb58d31-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:49.855562 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:49.855516 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:50.032153 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.032057 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:50.329776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.328984 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:50.585127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.585067 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:32:50.597981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.597954 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp" Oct 30 19:32:50.641649 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.641616 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" event={"ID":"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3","Type":"ContainerDied","Data":"363497b64f8822ae25ab63dca0365c985be2a0960d75df6abaa5b02bb648b6c4"} Oct 30 19:32:50.641816 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.641680 5627 generic.go:356] "Generic (PLEG): container finished" podID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerID="363497b64f8822ae25ab63dca0365c985be2a0960d75df6abaa5b02bb648b6c4" exitCode=0 Oct 30 19:32:50.646792 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.646760 5627 generic.go:356] "Generic (PLEG): container finished" podID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerID="62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439" exitCode=0 Oct 30 19:32:50.646929 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.646885 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" event={"ID":"b56101b9-7ef6-4d44-a239-e509c2c49ba3","Type":"ContainerDied","Data":"62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439"} Oct 30 19:32:50.646929 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.646911 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" Oct 30 19:32:50.647041 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.646933 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j" event={"ID":"b56101b9-7ef6-4d44-a239-e509c2c49ba3","Type":"ContainerDied","Data":"be910af3ff33ae72aae5a6c8d01a12d0e5b3483cb5e1f99735378984c178452e"} Oct 30 19:32:50.647041 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.646961 5627 scope.go:122] "RemoveContainer" containerID="62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439" Oct 30 19:32:50.654125 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.654103 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-certs\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.654336 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.654150 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-token\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-token\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.654336 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.654264 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-podinfo\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.654336 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.654310 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-credential-socket\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.654643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.654368 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-socket\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.654643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.654416 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istiod-ca-cert\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.654643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.654476 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-data\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.654643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.654615 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-envoy\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.656266 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.656020 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-kube-api-access-dsgb9\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-kube-api-access-dsgb9\") pod \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\" (UID: \"b56101b9-7ef6-4d44-a239-e509c2c49ba3\") " Oct 30 19:32:50.659003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.658977 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp"] Oct 30 19:32:50.661851 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.661833 5627 scope.go:122] "RemoveContainer" containerID="62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439" Oct 30 19:32:50.661941 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.661921 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-credential-socket" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.662263 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:32:50.662231 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439\": container with ID starting with 62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439 not found: ID does not exist" containerID="62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439" Oct 30 19:32:50.662367 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.662273 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439"} err="failed to get container status \"62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439\": rpc error: code = NotFound desc = could not find container \"62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439\": container with ID starting with 62a2f2ddec2ffec9bd52dea984c39ada460783163f9d6025db105eb43e2aa439 not found: ID does not exist" Oct 30 19:32:50.662367 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.662351 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-data" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.662595 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.662542 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-socket" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.662595 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.662605 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-certs" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.663132 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.663108 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istiod-ca-cert" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:32:50.665349 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.665315 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-podinfo" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:32:50.666333 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.666277 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-171808-5ddc5d69f6-44vrp"] Oct 30 19:32:50.667309 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.667284 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-token" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:50.671761 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.669617 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-kube-api-access-dsgb9" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "kube-api-access-dsgb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:50.671761 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.669895 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-envoy" pod "b56101b9-7ef6-4d44-a239-e509c2c49ba3" (UID: "b56101b9-7ef6-4d44-a239-e509c2c49ba3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.712556 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.712534 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:32:50.759426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759394 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.759426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759422 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsgb9\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-kube-api-access-dsgb9\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.759426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759434 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.759635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759443 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.759635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759451 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.759635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759460 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.759635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759468 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.759635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759475 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.759635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.759484 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56101b9-7ef6-4d44-a239-e509c2c49ba3-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.860063 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.859984 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-certs\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860063 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860038 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-data\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860311 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860095 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-podinfo\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860311 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860168 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-socket\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860311 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860200 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-token\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-token\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860311 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860235 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istiod-ca-cert\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860311 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860305 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-certs" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.860596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860324 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-kube-api-access-x7c7q\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-kube-api-access-x7c7q\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860348 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-data" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.860596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860433 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-envoy\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860480 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-credential-socket\") pod \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\" (UID: \"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3\") " Oct 30 19:32:50.860873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.860847 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-credential-socket" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.861363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.861336 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-socket" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.861693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.861673 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istiod-ca-cert" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:32:50.861850 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.861264 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.861850 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.861759 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.861850 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.861777 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.863175 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.863146 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-token" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:50.863264 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.863198 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-podinfo" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:32:50.863313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.863294 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-kube-api-access-x7c7q" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "kube-api-access-x7c7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:50.864722 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.864679 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-envoy" pod "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" (UID: "4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:50.919949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.919921 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:32:50.920095 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.919980 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:32:50.962917 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.962894 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.962917 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.962917 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.963060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.962927 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.963060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.962937 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.963060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.962945 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.963060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.962954 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7c7q\" (UniqueName: \"kubernetes.io/projected/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3-kube-api-access-x7c7q\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:50.974757 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.974733 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j"] Oct 30 19:32:50.980478 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.980457 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-lqp2j"] Oct 30 19:32:50.986626 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.986598 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x"] Oct 30 19:32:50.998397 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:50.998375 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r"] Oct 30 19:32:51.113993 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.113903 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:51.403848 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.403822 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="512cbb25-1c9d-4419-a15c-d954feb58d31" path="/var/lib/kubelet/pods/512cbb25-1c9d-4419-a15c-d954feb58d31/volumes" Oct 30 19:32:51.404353 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.404336 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" path="/var/lib/kubelet/pods/b56101b9-7ef6-4d44-a239-e509c2c49ba3/volumes" Oct 30 19:32:51.505805 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.505761 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:51.653979 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.653905 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" Oct 30 19:32:51.653979 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.653934 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd" event={"ID":"4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3","Type":"ContainerDied","Data":"4cbd702bac2d46b850d269b5bf6649fcdb928f41dbdd7f8d6aa004451d9d186d"} Oct 30 19:32:51.654202 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.653998 5627 scope.go:122] "RemoveContainer" containerID="363497b64f8822ae25ab63dca0365c985be2a0960d75df6abaa5b02bb648b6c4" Oct 30 19:32:51.681614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.681591 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd"] Oct 30 19:32:51.684127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:51.684108 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-gtgdd"] Oct 30 19:32:52.309415 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.309383 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" containerID="cri-o://b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69" gracePeriod=45 Oct 30 19:32:52.605377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.605343 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:32:52.616782 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.616754 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:52.668380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.667464 5627 generic.go:356] "Generic (PLEG): container finished" podID="f2127368-191b-4224-83c9-b43f8b95e582" containerID="b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69" exitCode=0 Oct 30 19:32:52.668380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.667523 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" Oct 30 19:32:52.668380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.667628 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" event={"ID":"f2127368-191b-4224-83c9-b43f8b95e582","Type":"ContainerDied","Data":"b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69"} Oct 30 19:32:52.668380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.667687 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb" event={"ID":"f2127368-191b-4224-83c9-b43f8b95e582","Type":"ContainerDied","Data":"02e6d76ad7f961c3de845d43d829adcf97f0c65b760677c4770db60ecf4388d8"} Oct 30 19:32:52.668380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.667758 5627 scope.go:122] "RemoveContainer" containerID="b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69" Oct 30 19:32:52.680955 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.680933 5627 scope.go:122] "RemoveContainer" containerID="b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69" Oct 30 19:32:52.681301 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:32:52.681268 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69\": container with ID starting with b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69 not found: ID does not exist" containerID="b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69" Oct 30 19:32:52.681404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.681305 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69"} err="failed to get container status \"b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69\": rpc error: code = NotFound desc = could not find container \"b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69\": container with ID starting with b0774a35be1632e29232da9d38dc8daf149aace850c809feff260c8fef9aee69 not found: ID does not exist" Oct 30 19:32:52.685793 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.685769 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-home\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-home\") pod \"f2127368-191b-4224-83c9-b43f8b95e582\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " Oct 30 19:32:52.685904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.685862 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-model-cache\") pod \"f2127368-191b-4224-83c9-b43f8b95e582\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " Oct 30 19:32:52.685904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.685899 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-tmp-dir\") pod \"f2127368-191b-4224-83c9-b43f8b95e582\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " Oct 30 19:32:52.686022 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.685965 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2127368-191b-4224-83c9-b43f8b95e582-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2127368-191b-4224-83c9-b43f8b95e582-tls-certs\") pod \"f2127368-191b-4224-83c9-b43f8b95e582\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " Oct 30 19:32:52.686022 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.685999 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2127368-191b-4224-83c9-b43f8b95e582-kube-api-access-nhr86\" (UniqueName: \"kubernetes.io/projected/f2127368-191b-4224-83c9-b43f8b95e582-kube-api-access-nhr86\") pod \"f2127368-191b-4224-83c9-b43f8b95e582\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " Oct 30 19:32:52.686022 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.686005 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-home" pod "f2127368-191b-4224-83c9-b43f8b95e582" (UID: "f2127368-191b-4224-83c9-b43f8b95e582"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:52.686174 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.686149 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-tmp-dir" pod "f2127368-191b-4224-83c9-b43f8b95e582" (UID: "f2127368-191b-4224-83c9-b43f8b95e582"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:52.686290 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.686271 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-model-cache" pod "f2127368-191b-4224-83c9-b43f8b95e582" (UID: "f2127368-191b-4224-83c9-b43f8b95e582"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:52.686290 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.686280 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-dshm\") pod \"f2127368-191b-4224-83c9-b43f8b95e582\" (UID: \"f2127368-191b-4224-83c9-b43f8b95e582\") " Oct 30 19:32:52.687460 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.687434 5627 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-home\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:52.687556 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.687465 5627 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-model-cache\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:52.687556 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.687480 5627 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-tmp-dir\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:52.691082 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.691056 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2127368-191b-4224-83c9-b43f8b95e582-tls-certs" pod "f2127368-191b-4224-83c9-b43f8b95e582" (UID: "f2127368-191b-4224-83c9-b43f8b95e582"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:32:52.691169 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.691107 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-dshm" pod "f2127368-191b-4224-83c9-b43f8b95e582" (UID: "f2127368-191b-4224-83c9-b43f8b95e582"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:52.691169 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.691146 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2127368-191b-4224-83c9-b43f8b95e582-kube-api-access-nhr86" pod "f2127368-191b-4224-83c9-b43f8b95e582" (UID: "f2127368-191b-4224-83c9-b43f8b95e582"). InnerVolumeSpecName "kube-api-access-nhr86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:52.789261 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.789228 5627 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2127368-191b-4224-83c9-b43f8b95e582-dshm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:52.789261 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.789255 5627 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2127368-191b-4224-83c9-b43f8b95e582-tls-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:52.789261 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.789267 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhr86\" (UniqueName: \"kubernetes.io/projected/f2127368-191b-4224-83c9-b43f8b95e582-kube-api-access-nhr86\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:52.803714 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.803661 5627 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:32:52.803858 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.803763 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:32:52.995469 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.995444 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb"] Oct 30 19:32:52.998261 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:52.998239 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/premium-w6-639c15-kserve-65f5bc9df9-p9dwb"] Oct 30 19:32:53.201202 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:53.201165 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln"] Oct 30 19:32:53.316305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:53.316273 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:53.399958 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:53.399928 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" path="/var/lib/kubelet/pods/4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3/volumes" Oct 30 19:32:53.400420 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:53.400406 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2127368-191b-4224-83c9-b43f8b95e582" path="/var/lib/kubelet/pods/f2127368-191b-4224-83c9-b43f8b95e582/volumes" Oct 30 19:32:53.616473 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:53.616399 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:54.006006 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:54.005974 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:54.508156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:54.508122 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:54.690814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:54.690788 5627 generic.go:356] "Generic (PLEG): container finished" podID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerID="f8a6670d890202195ef26b53d96e6304d480239b8402ff47ba11c1446b02c869" exitCode=0 Oct 30 19:32:54.691211 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:54.690869 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" event={"ID":"5be4340f-16bb-46ad-8117-4d31a32aa2c1","Type":"ContainerDied","Data":"f8a6670d890202195ef26b53d96e6304d480239b8402ff47ba11c1446b02c869"} Oct 30 19:32:54.692887 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:54.692860 5627 generic.go:356] "Generic (PLEG): container finished" podID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerID="d4348502a8d881932bf1ac17ebcd5e6458912364c576b49ac0d281bf59177315" exitCode=0 Oct 30 19:32:54.692997 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:54.692927 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" event={"ID":"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17","Type":"ContainerDied","Data":"d4348502a8d881932bf1ac17ebcd5e6458912364c576b49ac0d281bf59177315"} Oct 30 19:32:54.835782 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:54.835755 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:32:54.982061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:54.982023 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:55.033018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.032980 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-kube-api-access-j7qb4\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-kube-api-access-j7qb4\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.038209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.038172 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-socket\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.038424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.038319 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-certs\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.038424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.038354 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-token\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-token\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.038424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.038394 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istiod-ca-cert\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.038629 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.038450 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-data\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.038629 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.038514 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-podinfo\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.038629 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.038599 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-credential-socket\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.038629 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.038627 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-envoy\") pod \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\" (UID: \"5be4340f-16bb-46ad-8117-4d31a32aa2c1\") " Oct 30 19:32:55.040302 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.040028 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-credential-socket" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.040302 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.040170 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-certs" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.040302 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.040243 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-data" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.040302 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.040267 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-socket" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.041795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.041774 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istiod-ca-cert" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:32:55.043166 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.043117 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-token" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:55.059797 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.055898 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-podinfo" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:32:55.059797 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.056005 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-envoy" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.059797 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.057184 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-kube-api-access-j7qb4" pod "5be4340f-16bb-46ad-8117-4d31a32aa2c1" (UID: "5be4340f-16bb-46ad-8117-4d31a32aa2c1"). InnerVolumeSpecName "kube-api-access-j7qb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:55.102955 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.102913 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142794 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142826 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142842 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142856 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7qb4\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-kube-api-access-j7qb4\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142869 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142880 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142892 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142904 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.143049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.142914 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be4340f-16bb-46ad-8117-4d31a32aa2c1-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.262873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.262837 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:55.397247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.397159 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:55.435306 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.435268 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:32:55.448837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.448806 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-podinfo\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.449136 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.449116 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-certs\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.449283 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.449267 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-token\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-token\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.449416 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.449400 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-envoy\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.449584 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.449566 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-credential-socket\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.449761 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.449744 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-data\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.449908 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.449891 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-socket\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.450039 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.450025 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istiod-ca-cert\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.450176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.450160 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-kube-api-access-rqc6t\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-kube-api-access-rqc6t\") pod \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\" (UID: \"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17\") " Oct 30 19:32:55.450270 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.449443 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-certs" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.450334 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.450016 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-credential-socket" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.450334 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.450239 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-socket" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.450647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.450612 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istiod-ca-cert" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:32:55.451459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.451000 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-data" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.451459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.451373 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.451459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.451395 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.451459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.451410 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.451459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.451425 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.451459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.451437 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.453305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.453281 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-kube-api-access-rqc6t" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "kube-api-access-rqc6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:55.453741 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.453715 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-envoy" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:32:55.454562 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.454454 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-podinfo" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:32:55.481749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.478003 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-token" pod "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" (UID: "cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:32:55.552966 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.552927 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqc6t\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-kube-api-access-rqc6t\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.552966 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.552961 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.553167 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.552975 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.553167 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.552987 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:32:55.707740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.706565 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" Oct 30 19:32:55.707740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.706560 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg" event={"ID":"5be4340f-16bb-46ad-8117-4d31a32aa2c1","Type":"ContainerDied","Data":"ebdecf8f1fbd45ba19c18819a6cfe842e7333b73ca338190c02e885dc392877a"} Oct 30 19:32:55.707740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.706822 5627 scope.go:122] "RemoveContainer" containerID="f8a6670d890202195ef26b53d96e6304d480239b8402ff47ba11c1446b02c869" Oct 30 19:32:55.708495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.708319 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" event={"ID":"cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17","Type":"ContainerDied","Data":"f00dd2a04ca829b90d21a2e26769cd8525d30e5e9f9cb133236e753f6221f969"} Oct 30 19:32:55.708495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.708423 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf" Oct 30 19:32:55.720011 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.719989 5627 scope.go:122] "RemoveContainer" containerID="d4348502a8d881932bf1ac17ebcd5e6458912364c576b49ac0d281bf59177315" Oct 30 19:32:55.733364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.733334 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg"] Oct 30 19:32:55.737253 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.737223 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-grqdg"] Oct 30 19:32:55.755972 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.755940 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf"] Oct 30 19:32:55.760139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.760112 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-9cjqf"] Oct 30 19:32:55.894020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:55.893990 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:56.140006 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:56.139968 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:56.729896 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:56.729863 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:56.850252 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:56.850221 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:57.399022 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.398989 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" path="/var/lib/kubelet/pods/5be4340f-16bb-46ad-8117-4d31a32aa2c1/volumes" Oct 30 19:32:57.399548 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.399527 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" path="/var/lib/kubelet/pods/cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17/volumes" Oct 30 19:32:57.629803 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.629775 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:57.745666 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.745629 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:57.865177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.865132 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:57.936122 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.935783 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8"] Oct 30 19:32:57.936122 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.936014 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" containerID="cri-o://66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a" gracePeriod=30 Oct 30 19:32:57.943007 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.942968 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2"] Oct 30 19:32:57.944019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.943983 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" containerID="cri-o://48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7" gracePeriod=30 Oct 30 19:32:57.966872 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.966830 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5"] Oct 30 19:32:57.967139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:57.967110 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" containerID="cri-o://72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77" gracePeriod=30 Oct 30 19:32:58.098782 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:58.097730 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:58.509880 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:58.509842 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv"] Oct 30 19:32:58.612871 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:58.612665 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:58.741392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:58.741359 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:59.032209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:59.032175 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:59.923807 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:32:59.923767 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:00.826131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:00.826070 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:01.714896 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:01.714870 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:02.294874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:02.294839 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:02.473728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:02.464421 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:02.719902 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:02.719821 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:02.913765 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:02.913735 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:03.140353 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.140319 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:03.285655 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.285632 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:33:03.289274 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.289255 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:33:03.292958 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.292933 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:33:03.317754 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.317728 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:03.367247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.367215 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-podinfo\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.367471 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.367454 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-socket\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.367583 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.367570 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-certs\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.367747 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.367732 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46d47944-f78b-45f4-a00e-094ddb1ce717-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d47944-f78b-45f4-a00e-094ddb1ce717-istiod-ca-cert\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.367875 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.367861 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/568a6fd2-2a37-40cf-a335-be350847e9ea-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/568a6fd2-2a37-40cf-a335-be350847e9ea-istiod-ca-cert\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.367998 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.367983 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-envoy\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.368125 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368110 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-envoy\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.368271 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368250 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-certs\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.368350 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368302 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-socket\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.368350 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368340 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-kube-api-access-4zjtm\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-kube-api-access-4zjtm\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.368466 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368369 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-data\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.368466 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368424 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-socket\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.368569 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368494 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-data\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.368569 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368550 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-podinfo\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.368674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368599 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-token\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-token\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.368674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368639 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-credential-socket\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.368843 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368720 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-token\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-token\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.368843 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368757 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-kube-api-access-zlhr9\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-kube-api-access-zlhr9\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.368843 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368116 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-certs" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.368843 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368799 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-certs\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.369046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368846 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-podinfo\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.369046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368883 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-envoy\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.369046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368922 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-data\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.369046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368986 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-kube-api-access-wfjxz\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-kube-api-access-wfjxz\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.369241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.369050 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbe1332a-d689-4113-a3a5-b16e82f1f167-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe1332a-d689-4113-a3a5-b16e82f1f167-istiod-ca-cert\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.369241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.369106 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-credential-socket\") pod \"fbe1332a-d689-4113-a3a5-b16e82f1f167\" (UID: \"fbe1332a-d689-4113-a3a5-b16e82f1f167\") " Oct 30 19:33:03.369241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.369142 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-token\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-token\") pod \"568a6fd2-2a37-40cf-a335-be350847e9ea\" (UID: \"568a6fd2-2a37-40cf-a335-be350847e9ea\") " Oct 30 19:33:03.369241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.369184 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-credential-socket\") pod \"46d47944-f78b-45f4-a00e-094ddb1ce717\" (UID: \"46d47944-f78b-45f4-a00e-094ddb1ce717\") " Oct 30 19:33:03.370330 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.370303 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.372855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368138 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-socket" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.372956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368508 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568a6fd2-2a37-40cf-a335-be350847e9ea-istiod-ca-cert" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:03.372956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368776 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-certs" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.372956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.368872 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46d47944-f78b-45f4-a00e-094ddb1ce717-istiod-ca-cert" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:03.372956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.371924 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-podinfo" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:03.372956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.372056 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-socket" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.372956 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.372500 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-credential-socket" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.373223 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.373189 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-envoy" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.373352 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.373329 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-certs" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.373352 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.373336 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-podinfo" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:03.373626 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.373600 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-data" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.373763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.373688 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbe1332a-d689-4113-a3a5-b16e82f1f167-istiod-ca-cert" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:03.373960 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.373940 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-data" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.374262 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.374238 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-socket" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.374511 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.374496 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-data" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.374683 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.374667 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-kube-api-access-zlhr9" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "kube-api-access-zlhr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:03.375072 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.375048 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-credential-socket" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.375147 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.375110 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-envoy" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.375202 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.375187 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-kube-api-access-4zjtm" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "kube-api-access-4zjtm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:03.375387 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.375363 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-credential-socket" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.379387 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.379293 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-token" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:03.379387 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.379340 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-token" pod "fbe1332a-d689-4113-a3a5-b16e82f1f167" (UID: "fbe1332a-d689-4113-a3a5-b16e82f1f167"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:03.379684 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.379660 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-envoy" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:03.380061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.380016 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-token" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:03.388085 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.380239 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-kube-api-access-wfjxz" pod "46d47944-f78b-45f4-a00e-094ddb1ce717" (UID: "46d47944-f78b-45f4-a00e-094ddb1ce717"). InnerVolumeSpecName "kube-api-access-wfjxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:03.388085 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.382842 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-podinfo" pod "568a6fd2-2a37-40cf-a335-be350847e9ea" (UID: "568a6fd2-2a37-40cf-a335-be350847e9ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:03.471284 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471260 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471284 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471284 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471293 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471302 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zjtm\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-kube-api-access-4zjtm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471311 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471320 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471328 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471335 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471343 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471351 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471359 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471367 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlhr9\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-kube-api-access-zlhr9\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471376 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471384 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471392 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471400 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471408 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfjxz\" (UniqueName: \"kubernetes.io/projected/46d47944-f78b-45f4-a00e-094ddb1ce717-kube-api-access-wfjxz\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471416 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe1332a-d689-4113-a3a5-b16e82f1f167-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471424 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471431 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/568a6fd2-2a37-40cf-a335-be350847e9ea-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471440 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471448 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471457 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d47944-f78b-45f4-a00e-094ddb1ce717-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471466 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d47944-f78b-45f4-a00e-094ddb1ce717-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471475 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/568a6fd2-2a37-40cf-a335-be350847e9ea-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.471470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.471483 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe1332a-d689-4113-a3a5-b16e82f1f167-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:03.510810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.510757 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" podUID="1668db65-90fe-4e2f-9ae9-70c50a2c788a" containerName="payload-processing" containerID="cri-o://fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43" gracePeriod=25 Oct 30 19:33:03.762451 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.762368 5627 generic.go:356] "Generic (PLEG): container finished" podID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerID="48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7" exitCode=0 Oct 30 19:33:03.762451 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.762433 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" Oct 30 19:33:03.762669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.762451 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" event={"ID":"fbe1332a-d689-4113-a3a5-b16e82f1f167","Type":"ContainerDied","Data":"48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7"} Oct 30 19:33:03.762669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.762498 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2" event={"ID":"fbe1332a-d689-4113-a3a5-b16e82f1f167","Type":"ContainerDied","Data":"305b2821e3b69425c0ed2189e766cb2034e3fcd4ef6916ff99dfbf6d21a901c1"} Oct 30 19:33:03.762669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.762527 5627 scope.go:122] "RemoveContainer" containerID="48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7" Oct 30 19:33:03.764020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.763996 5627 generic.go:356] "Generic (PLEG): container finished" podID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerID="66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a" exitCode=0 Oct 30 19:33:03.764133 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.764071 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" Oct 30 19:33:03.764133 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.764092 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" event={"ID":"568a6fd2-2a37-40cf-a335-be350847e9ea","Type":"ContainerDied","Data":"66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a"} Oct 30 19:33:03.764249 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.764163 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8" event={"ID":"568a6fd2-2a37-40cf-a335-be350847e9ea","Type":"ContainerDied","Data":"076a1278d4d5bc2b8bf274e5e4634a9b19c4d40f733256fb82dcb356934b3bed"} Oct 30 19:33:03.765662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.765643 5627 generic.go:356] "Generic (PLEG): container finished" podID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerID="72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77" exitCode=0 Oct 30 19:33:03.765800 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.765669 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" event={"ID":"46d47944-f78b-45f4-a00e-094ddb1ce717","Type":"ContainerDied","Data":"72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77"} Oct 30 19:33:03.765800 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.765737 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" Oct 30 19:33:03.765900 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.765739 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5" event={"ID":"46d47944-f78b-45f4-a00e-094ddb1ce717","Type":"ContainerDied","Data":"d9aa02b7dc3899c87d99ced86c96b2b384975680aec6eba308d14e3e86544928"} Oct 30 19:33:03.774543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.774522 5627 scope.go:122] "RemoveContainer" containerID="48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7" Oct 30 19:33:03.774806 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:03.774786 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7\": container with ID starting with 48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7 not found: ID does not exist" containerID="48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7" Oct 30 19:33:03.774874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.774814 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7"} err="failed to get container status \"48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7\": rpc error: code = NotFound desc = could not find container \"48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7\": container with ID starting with 48918facfb501ab74a0354a6c76af2ee7da971c490c209f471ac53b2aea496c7 not found: ID does not exist" Oct 30 19:33:03.774874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.774831 5627 scope.go:122] "RemoveContainer" containerID="66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a" Oct 30 19:33:03.784325 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.784306 5627 scope.go:122] "RemoveContainer" containerID="66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a" Oct 30 19:33:03.784604 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:03.784577 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a\": container with ID starting with 66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a not found: ID does not exist" containerID="66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a" Oct 30 19:33:03.784663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.784601 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a"} err="failed to get container status \"66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a\": rpc error: code = NotFound desc = could not find container \"66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a\": container with ID starting with 66fb056d610a74e46fc79c21718a0d668ac8c17da3ad73d6cb98372c736dc69a not found: ID does not exist" Oct 30 19:33:03.784663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.784616 5627 scope.go:122] "RemoveContainer" containerID="72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77" Oct 30 19:33:03.793409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.793388 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5"] Oct 30 19:33:03.794052 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.794036 5627 scope.go:122] "RemoveContainer" containerID="72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77" Oct 30 19:33:03.794325 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:03.794307 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77\": container with ID starting with 72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77 not found: ID does not exist" containerID="72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77" Oct 30 19:33:03.794412 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.794331 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77"} err="failed to get container status \"72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77\": rpc error: code = NotFound desc = could not find container \"72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77\": container with ID starting with 72cb64e95a5069dfbb9a6634f1103bfa6e371f0f7cffc000c9250a194cb8ff77 not found: ID does not exist" Oct 30 19:33:03.798833 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.798815 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-hqfq5"] Oct 30 19:33:03.811810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.811786 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8"] Oct 30 19:33:03.815714 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.815681 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-qtzj8"] Oct 30 19:33:03.831814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.831793 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2"] Oct 30 19:33:03.839377 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.839358 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-ckxz2"] Oct 30 19:33:03.908682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:03.908654 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:04.613940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:04.613910 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:04.997560 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:04.997528 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:05.140020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:05.139992 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:05.320264 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:05.320180 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:05.404134 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:05.404104 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" path="/var/lib/kubelet/pods/46d47944-f78b-45f4-a00e-094ddb1ce717/volumes" Oct 30 19:33:05.404775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:05.404753 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" path="/var/lib/kubelet/pods/568a6fd2-2a37-40cf-a335-be350847e9ea/volumes" Oct 30 19:33:05.405330 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:05.405313 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" path="/var/lib/kubelet/pods/fbe1332a-d689-4113-a3a5-b16e82f1f167/volumes" Oct 30 19:33:05.486154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:05.486126 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:05.687163 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:05.687080 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:06.009899 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.009865 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" containerID="cri-o://4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4" gracePeriod=45 Oct 30 19:33:06.024184 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.024157 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" containerID="cri-o://0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143" gracePeriod=45 Oct 30 19:33:06.289212 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.289181 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:33:06.311198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.311175 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:33:06.427804 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.427776 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tmp-dir\") pod \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " Oct 30 19:33:06.428001 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.427867 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09f38231-eeaf-48ad-b3d4-cfe1d497418c-kube-api-access-gp6ng\" (UniqueName: \"kubernetes.io/projected/09f38231-eeaf-48ad-b3d4-cfe1d497418c-kube-api-access-gp6ng\") pod \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " Oct 30 19:33:06.428001 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.427900 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tmp-dir\") pod \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " Oct 30 19:33:06.428001 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.427935 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-home\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-home\") pod \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " Oct 30 19:33:06.428166 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428025 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-dshm\") pod \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " Oct 30 19:33:06.428166 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428092 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tls-certs\" (UniqueName: \"kubernetes.io/secret/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tls-certs\") pod \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " Oct 30 19:33:06.428166 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428141 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-model-cache\") pod \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " Oct 30 19:33:06.428324 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428175 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-model-cache\") pod \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " Oct 30 19:33:06.428324 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428208 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-dshm\") pod \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " Oct 30 19:33:06.428324 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428255 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-kube-api-access-85tn5\" (UniqueName: \"kubernetes.io/projected/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-kube-api-access-85tn5\") pod \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " Oct 30 19:33:06.428324 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428295 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-home\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-home\") pod \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\" (UID: \"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3\") " Oct 30 19:33:06.428540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428362 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-home" pod "09f38231-eeaf-48ad-b3d4-cfe1d497418c" (UID: "09f38231-eeaf-48ad-b3d4-cfe1d497418c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:06.428540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428390 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tls-certs\" (UniqueName: \"kubernetes.io/secret/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tls-certs\") pod \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\" (UID: \"09f38231-eeaf-48ad-b3d4-cfe1d497418c\") " Oct 30 19:33:06.428643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.428552 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-model-cache" pod "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" (UID: "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:06.429499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.429032 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-model-cache" pod "09f38231-eeaf-48ad-b3d4-cfe1d497418c" (UID: "09f38231-eeaf-48ad-b3d4-cfe1d497418c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:06.429499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.429275 5627 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-home\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.429499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.429295 5627 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-model-cache\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.429499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.429311 5627 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-model-cache\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.429837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.429528 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tmp-dir" pod "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" (UID: "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:06.429837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.429781 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tmp-dir" pod "09f38231-eeaf-48ad-b3d4-cfe1d497418c" (UID: "09f38231-eeaf-48ad-b3d4-cfe1d497418c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:06.430019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.429986 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-home" pod "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" (UID: "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:06.431067 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.431040 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f38231-eeaf-48ad-b3d4-cfe1d497418c-kube-api-access-gp6ng" pod "09f38231-eeaf-48ad-b3d4-cfe1d497418c" (UID: "09f38231-eeaf-48ad-b3d4-cfe1d497418c"). InnerVolumeSpecName "kube-api-access-gp6ng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:06.432075 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.432040 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tls-certs" pod "09f38231-eeaf-48ad-b3d4-cfe1d497418c" (UID: "09f38231-eeaf-48ad-b3d4-cfe1d497418c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:06.433151 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.433131 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-dshm" pod "09f38231-eeaf-48ad-b3d4-cfe1d497418c" (UID: "09f38231-eeaf-48ad-b3d4-cfe1d497418c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:06.433734 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.433671 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tls-certs" pod "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" (UID: "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:06.433900 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.433880 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-kube-api-access-85tn5" pod "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" (UID: "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3"). InnerVolumeSpecName "kube-api-access-85tn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:06.435174 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.435151 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-dshm" pod "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" (UID: "57d48892-90cc-4a09-80f8-3bd0d2c3a7b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:06.530856 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530742 5627 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tls-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.530856 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530814 5627 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-tmp-dir\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.530856 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530831 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp6ng\" (UniqueName: \"kubernetes.io/projected/09f38231-eeaf-48ad-b3d4-cfe1d497418c-kube-api-access-gp6ng\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.531156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530862 5627 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tmp-dir\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.531156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530877 5627 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f38231-eeaf-48ad-b3d4-cfe1d497418c-dshm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.531156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530888 5627 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-tls-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.531156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530899 5627 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-dshm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.531156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530918 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85tn5\" (UniqueName: \"kubernetes.io/projected/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-kube-api-access-85tn5\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.531156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.530930 5627 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3-home\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:06.623352 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.623292 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:06.787234 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.787154 5627 generic.go:356] "Generic (PLEG): container finished" podID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerID="4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4" exitCode=0 Oct 30 19:33:06.787234 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.787215 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" Oct 30 19:33:06.787662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.787233 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" event={"ID":"09f38231-eeaf-48ad-b3d4-cfe1d497418c","Type":"ContainerDied","Data":"4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4"} Oct 30 19:33:06.787662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.787279 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x" event={"ID":"09f38231-eeaf-48ad-b3d4-cfe1d497418c","Type":"ContainerDied","Data":"e7365037f21a937fce7f91dbc366973ce8251a9ee6fd6cccb2d882fa57203621"} Oct 30 19:33:06.787662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.787297 5627 scope.go:122] "RemoveContainer" containerID="4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4" Oct 30 19:33:06.788887 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.788860 5627 generic.go:356] "Generic (PLEG): container finished" podID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerID="0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143" exitCode=0 Oct 30 19:33:06.789003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.788918 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" Oct 30 19:33:06.789003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.788929 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" event={"ID":"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3","Type":"ContainerDied","Data":"0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143"} Oct 30 19:33:06.789003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.788958 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" event={"ID":"57d48892-90cc-4a09-80f8-3bd0d2c3a7b3","Type":"ContainerDied","Data":"c85efe0529b67010a7948d100ba5db62cef8710ce0e9c34cb5b32948a33d38fc"} Oct 30 19:33:06.798195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.798180 5627 scope.go:122] "RemoveContainer" containerID="4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4" Oct 30 19:33:06.798490 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:06.798462 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4\": container with ID starting with 4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4 not found: ID does not exist" containerID="4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4" Oct 30 19:33:06.798596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.798497 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4"} err="failed to get container status \"4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4\": rpc error: code = NotFound desc = could not find container \"4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4\": container with ID starting with 4ce1ee3ba2d6312d87d3c6125f4ab70ff93ddf1423af6362363cdf6f309eb1c4 not found: ID does not exist" Oct 30 19:33:06.798596 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.798520 5627 scope.go:122] "RemoveContainer" containerID="0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143" Oct 30 19:33:06.807038 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.807020 5627 scope.go:122] "RemoveContainer" containerID="0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143" Oct 30 19:33:06.807288 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:06.807270 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143\": container with ID starting with 0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143 not found: ID does not exist" containerID="0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143" Oct 30 19:33:06.807328 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.807296 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143"} err="failed to get container status \"0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143\": rpc error: code = NotFound desc = could not find container \"0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143\": container with ID starting with 0bf724045823f304aba2f3d0dba31081d818c203301a5770f24630e9c4092143 not found: ID does not exist" Oct 30 19:33:06.817916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.817895 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x"] Oct 30 19:33:06.822623 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.822604 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/premium-w5-f7f075-kserve-5dbfc955fd-nkz4x"] Oct 30 19:33:06.835185 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.835156 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r"] Oct 30 19:33:06.841653 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:06.841636 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r"] Oct 30 19:33:07.286475 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:07.286438 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/simulator-w5-f7f075-kserve-7b9946f8bc-p8k4r" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded" Oct 30 19:33:07.382545 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:07.382502 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" podUID="1668db65-90fe-4e2f-9ae9-70c50a2c788a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:33:07.399523 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:07.399493 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" path="/var/lib/kubelet/pods/09f38231-eeaf-48ad-b3d4-cfe1d497418c/volumes" Oct 30 19:33:07.400029 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:07.400011 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" path="/var/lib/kubelet/pods/57d48892-90cc-4a09-80f8-3bd0d2c3a7b3/volumes" Oct 30 19:33:07.432877 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:07.432854 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:07.777635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:07.777602 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:08.224866 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.224777 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" containerID="cri-o://e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb" gracePeriod=45 Oct 30 19:33:08.339154 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.339121 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:08.367519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.367483 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": dial tcp 10.133.0.48:8000: connect: connection refused" Oct 30 19:33:08.490920 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.490897 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:33:08.667668 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.667633 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-home\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-home\") pod \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " Oct 30 19:33:08.667867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.667722 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tls-certs\" (UniqueName: \"kubernetes.io/secret/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tls-certs\") pod \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " Oct 30 19:33:08.667867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.667798 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tmp-dir\") pod \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " Oct 30 19:33:08.667867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.667835 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-dshm\") pod \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " Oct 30 19:33:08.668106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.667887 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-model-cache\") pod \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " Oct 30 19:33:08.668106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.667930 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-home" pod "c35eab54-26ba-4fd5-958f-1e69cb47aae0" (UID: "c35eab54-26ba-4fd5-958f-1e69cb47aae0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:08.668106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.668013 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c35eab54-26ba-4fd5-958f-1e69cb47aae0-kube-api-access-xmwtn\" (UniqueName: \"kubernetes.io/projected/c35eab54-26ba-4fd5-958f-1e69cb47aae0-kube-api-access-xmwtn\") pod \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\" (UID: \"c35eab54-26ba-4fd5-958f-1e69cb47aae0\") " Oct 30 19:33:08.668511 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.668295 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tmp-dir" pod "c35eab54-26ba-4fd5-958f-1e69cb47aae0" (UID: "c35eab54-26ba-4fd5-958f-1e69cb47aae0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:08.668511 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.668451 5627 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-home\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:08.668511 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.668464 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-model-cache" pod "c35eab54-26ba-4fd5-958f-1e69cb47aae0" (UID: "c35eab54-26ba-4fd5-958f-1e69cb47aae0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:08.668511 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.668475 5627 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tmp-dir\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:08.670651 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.670624 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-dshm" pod "c35eab54-26ba-4fd5-958f-1e69cb47aae0" (UID: "c35eab54-26ba-4fd5-958f-1e69cb47aae0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:08.670817 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.670722 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tls-certs" pod "c35eab54-26ba-4fd5-958f-1e69cb47aae0" (UID: "c35eab54-26ba-4fd5-958f-1e69cb47aae0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:08.670990 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.670964 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c35eab54-26ba-4fd5-958f-1e69cb47aae0-kube-api-access-xmwtn" pod "c35eab54-26ba-4fd5-958f-1e69cb47aae0" (UID: "c35eab54-26ba-4fd5-958f-1e69cb47aae0"). InnerVolumeSpecName "kube-api-access-xmwtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:08.773909 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.773819 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmwtn\" (UniqueName: \"kubernetes.io/projected/c35eab54-26ba-4fd5-958f-1e69cb47aae0-kube-api-access-xmwtn\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:08.773909 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.773856 5627 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c35eab54-26ba-4fd5-958f-1e69cb47aae0-tls-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:08.773909 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.773866 5627 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-dshm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:08.773909 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.773875 5627 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c35eab54-26ba-4fd5-958f-1e69cb47aae0-model-cache\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:08.803967 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.803933 5627 generic.go:356] "Generic (PLEG): container finished" podID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerID="e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb" exitCode=0 Oct 30 19:33:08.804139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.803974 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" Oct 30 19:33:08.804139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.804014 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" event={"ID":"c35eab54-26ba-4fd5-958f-1e69cb47aae0","Type":"ContainerDied","Data":"e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb"} Oct 30 19:33:08.804139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.804057 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln" event={"ID":"c35eab54-26ba-4fd5-958f-1e69cb47aae0","Type":"ContainerDied","Data":"9368c068f9efe70aed6061c5e0b00b0f88d1942f1a3ffbcc3053d9a4962dc347"} Oct 30 19:33:08.804139 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.804076 5627 scope.go:122] "RemoveContainer" containerID="e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb" Oct 30 19:33:08.814678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.814661 5627 scope.go:122] "RemoveContainer" containerID="e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb" Oct 30 19:33:08.814989 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:08.814968 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb\": container with ID starting with e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb not found: ID does not exist" containerID="e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb" Oct 30 19:33:08.815082 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.814995 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb"} err="failed to get container status \"e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb\": rpc error: code = NotFound desc = could not find container \"e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb\": container with ID starting with e4509ee5036f394ea9206c6531ccef8a5c6760d391ec82335cc8aeb5e5d474fb not found: ID does not exist" Oct 30 19:33:08.832108 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.832082 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln"] Oct 30 19:33:08.836415 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:08.836386 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/simulator-w1-6cf459-kserve-649fc74bcc-xq7ln"] Oct 30 19:33:09.177314 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:09.177233 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:09.405899 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:09.405863 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" path="/var/lib/kubelet/pods/c35eab54-26ba-4fd5-958f-1e69cb47aae0/volumes" Oct 30 19:33:10.038734 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:10.038297 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:10.445814 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:10.444004 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:33:11.033473 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:11.033431 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt"] Oct 30 19:33:11.035107 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:11.035072 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" podUID="71a3c20c-9338-4bce-a358-33a1d752ff46" containerName="maas-api" containerID="cri-o://5f3d3abe4893cac10b282d9f871c1a6a19317cd1fea0c7e70874f9bf7bd14fc8" gracePeriod=30 Oct 30 19:33:11.168401 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:11.168359 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:11.494828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:11.489216 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:11.876811 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:11.825689 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:11.904760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:11.876995 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws"] Oct 30 19:33:11.904760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:11.877226 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" podUID="e8d59d58-fa19-4884-9d5a-5583b2758491" containerName="maas-api" containerID="cri-o://28c6e970cf241bc91071e23996c86626421d5eeced8f311c6444bc5ab2ccda83" gracePeriod=30 Oct 30 19:33:12.049771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.031754 5627 generic.go:356] "Generic (PLEG): container finished" podID="71a3c20c-9338-4bce-a358-33a1d752ff46" containerID="5f3d3abe4893cac10b282d9f871c1a6a19317cd1fea0c7e70874f9bf7bd14fc8" exitCode=0 Oct 30 19:33:12.049771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.031908 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" event={"ID":"71a3c20c-9338-4bce-a358-33a1d752ff46","Type":"ContainerDied","Data":"5f3d3abe4893cac10b282d9f871c1a6a19317cd1fea0c7e70874f9bf7bd14fc8"} Oct 30 19:33:12.153778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.139032 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:12.320738 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.318164 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:12.453098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.452836 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:12.674730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.671352 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6"] Oct 30 19:33:12.674730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.671630 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" containerID="cri-o://c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea" gracePeriod=30 Oct 30 19:33:12.743739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.742932 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m"] Oct 30 19:33:12.743739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.743016 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv"] Oct 30 19:33:12.743739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.743262 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" containerID="cri-o://b4a355e5d99834716773729fe8d02336f1c98baf0677b8ecb44f2ac896d23dff" gracePeriod=30 Oct 30 19:33:12.759733 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.744666 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" containerID="cri-o://9cdda0f05c77105a42ac39a01eb692a892ebd4e0d62eb9f7d07ae887f292f727" gracePeriod=30 Oct 30 19:33:12.780455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.780421 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:12.995968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:12.988016 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:33:13.057607 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.057576 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:13.191991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.191798 5627 generic.go:356] "Generic (PLEG): container finished" podID="e8d59d58-fa19-4884-9d5a-5583b2758491" containerID="28c6e970cf241bc91071e23996c86626421d5eeced8f311c6444bc5ab2ccda83" exitCode=0 Oct 30 19:33:13.191991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.191915 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" event={"ID":"e8d59d58-fa19-4884-9d5a-5583b2758491","Type":"ContainerDied","Data":"28c6e970cf241bc91071e23996c86626421d5eeced8f311c6444bc5ab2ccda83"} Oct 30 19:33:13.222731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.218038 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:33:13.242974 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.242742 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-maas-api-tls\") pod \"71a3c20c-9338-4bce-a358-33a1d752ff46\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " Oct 30 19:33:13.242974 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.242871 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71a3c20c-9338-4bce-a358-33a1d752ff46-kube-api-access-k7jhz\" (UniqueName: \"kubernetes.io/projected/71a3c20c-9338-4bce-a358-33a1d752ff46-kube-api-access-k7jhz\") pod \"71a3c20c-9338-4bce-a358-33a1d752ff46\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " Oct 30 19:33:13.242974 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.242975 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71a3c20c-9338-4bce-a358-33a1d752ff46-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a3c20c-9338-4bce-a358-33a1d752ff46-openshift-service-ca\") pod \"71a3c20c-9338-4bce-a358-33a1d752ff46\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " Oct 30 19:33:13.243273 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.243030 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-metrics-certs\") pod \"71a3c20c-9338-4bce-a358-33a1d752ff46\" (UID: \"71a3c20c-9338-4bce-a358-33a1d752ff46\") " Oct 30 19:33:13.319971 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.318834 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71a3c20c-9338-4bce-a358-33a1d752ff46-openshift-service-ca" pod "71a3c20c-9338-4bce-a358-33a1d752ff46" (UID: "71a3c20c-9338-4bce-a358-33a1d752ff46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:13.449621 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.441612 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a3c20c-9338-4bce-a358-33a1d752ff46-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:13.544842 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.540659 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:33:13.547397 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.547369 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-metrics-certs" pod "71a3c20c-9338-4bce-a358-33a1d752ff46" (UID: "71a3c20c-9338-4bce-a358-33a1d752ff46"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:13.557224 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.557190 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a3c20c-9338-4bce-a358-33a1d752ff46-kube-api-access-k7jhz" pod "71a3c20c-9338-4bce-a358-33a1d752ff46" (UID: "71a3c20c-9338-4bce-a358-33a1d752ff46"). InnerVolumeSpecName "kube-api-access-k7jhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:13.557796 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.557478 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-maas-api-tls" pod "71a3c20c-9338-4bce-a358-33a1d752ff46" (UID: "71a3c20c-9338-4bce-a358-33a1d752ff46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:13.568142 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.568048 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:13.654515 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.654424 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls\") pod \"e8d59d58-fa19-4884-9d5a-5583b2758491\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " Oct 30 19:33:13.655292 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.655257 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-metrics-certs\") pod \"e8d59d58-fa19-4884-9d5a-5583b2758491\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " Oct 30 19:33:13.655898 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.655881 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8d59d58-fa19-4884-9d5a-5583b2758491-kube-api-access-p9jp9\" (UniqueName: \"kubernetes.io/projected/e8d59d58-fa19-4884-9d5a-5583b2758491-kube-api-access-p9jp9\") pod \"e8d59d58-fa19-4884-9d5a-5583b2758491\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " Oct 30 19:33:13.656086 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.656062 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8d59d58-fa19-4884-9d5a-5583b2758491-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d59d58-fa19-4884-9d5a-5583b2758491-openshift-service-ca\") pod \"e8d59d58-fa19-4884-9d5a-5583b2758491\" (UID: \"e8d59d58-fa19-4884-9d5a-5583b2758491\") " Oct 30 19:33:13.658651 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.658578 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d59d58-fa19-4884-9d5a-5583b2758491-openshift-service-ca" pod "e8d59d58-fa19-4884-9d5a-5583b2758491" (UID: "e8d59d58-fa19-4884-9d5a-5583b2758491"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:13.661408 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.661379 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:13.662034 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.662020 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7jhz\" (UniqueName: \"kubernetes.io/projected/71a3c20c-9338-4bce-a358-33a1d752ff46-kube-api-access-k7jhz\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:13.662226 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.662190 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d59d58-fa19-4884-9d5a-5583b2758491-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:13.662354 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.662341 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a3c20c-9338-4bce-a358-33a1d752ff46-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:13.698041 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.695894 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls" pod "e8d59d58-fa19-4884-9d5a-5583b2758491" (UID: "e8d59d58-fa19-4884-9d5a-5583b2758491"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:13.698041 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.697837 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-metrics-certs" pod "e8d59d58-fa19-4884-9d5a-5583b2758491" (UID: "e8d59d58-fa19-4884-9d5a-5583b2758491"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:13.707340 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.699397 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8d59d58-fa19-4884-9d5a-5583b2758491-kube-api-access-p9jp9" pod "e8d59d58-fa19-4884-9d5a-5583b2758491" (UID: "e8d59d58-fa19-4884-9d5a-5583b2758491"). InnerVolumeSpecName "kube-api-access-p9jp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:13.766078 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.764626 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:13.766078 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.764675 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8d59d58-fa19-4884-9d5a-5583b2758491-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:13.766078 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:13.764694 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9jp9\" (UniqueName: \"kubernetes.io/projected/e8d59d58-fa19-4884-9d5a-5583b2758491-kube-api-access-p9jp9\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:14.062820 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.062641 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:14.247728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.244927 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:14.290917 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.283694 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" event={"ID":"71a3c20c-9338-4bce-a358-33a1d752ff46","Type":"ContainerDied","Data":"b322597f5d22cc0854e1fa6a40049ee736c357c60236f99e555ffe85953b6dc0"} Oct 30 19:33:14.291410 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.291379 5627 scope.go:122] "RemoveContainer" containerID="5f3d3abe4893cac10b282d9f871c1a6a19317cd1fea0c7e70874f9bf7bd14fc8" Oct 30 19:33:14.291897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.291876 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt" Oct 30 19:33:14.309830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.308882 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" event={"ID":"e8d59d58-fa19-4884-9d5a-5583b2758491","Type":"ContainerDied","Data":"a886782596804cc793e877246484765d4b26857322509222ee421d918853232b"} Oct 30 19:33:14.309830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.309039 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws" Oct 30 19:33:14.363072 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.356913 5627 scope.go:122] "RemoveContainer" containerID="28c6e970cf241bc91071e23996c86626421d5eeced8f311c6444bc5ab2ccda83" Oct 30 19:33:14.454789 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.450776 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws"] Oct 30 19:33:14.460763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.455021 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-6cf459-5796d76b95-fdbws"] Oct 30 19:33:14.571781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.556745 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt"] Oct 30 19:33:14.571781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.559418 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:14.571781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:14.562198 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f7f075-5cb964688f-t6fdt"] Oct 30 19:33:15.138681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:15.138610 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:15.400232 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:15.400149 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71a3c20c-9338-4bce-a358-33a1d752ff46" path="/var/lib/kubelet/pods/71a3c20c-9338-4bce-a358-33a1d752ff46/volumes" Oct 30 19:33:15.400651 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:15.400637 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8d59d58-fa19-4884-9d5a-5583b2758491" path="/var/lib/kubelet/pods/e8d59d58-fa19-4884-9d5a-5583b2758491/volumes" Oct 30 19:33:15.708565 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:15.708483 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:15.833425 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:15.833394 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:16.121428 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:16.121398 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:16.936566 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:16.936528 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:17.383826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:17.383791 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" podUID="1668db65-90fe-4e2f-9ae9-70c50a2c788a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:33:17.711043 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:17.710951 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:17.956684 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:17.956565 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:18.114050 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.114027 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:33:18.192951 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.192922 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-certs\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193091 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.192995 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-envoy\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193091 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193026 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-token\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193091 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193045 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-credential-socket\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193094 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istiod-ca-cert\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193139 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-socket\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193204 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-kube-api-access-wh5f9\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-kube-api-access-wh5f9\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193206 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-certs" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.193277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193261 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-data\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193318 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-podinfo\") pod \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\" (UID: \"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2\") " Oct 30 19:33:18.193909 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193887 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.194057 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.193951 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-socket" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.194157 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.194099 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-data" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.194256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.194164 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-credential-socket" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.194335 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.194274 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istiod-ca-cert" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:18.195450 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.195427 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-token" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:18.195947 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.195928 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-envoy" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.196161 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.196136 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-kube-api-access-wh5f9" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "kube-api-access-wh5f9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:18.197137 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.197117 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-podinfo" pod "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" (UID: "c2e5a633-4a42-4a8d-9eae-c7805c2cdab2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:18.295176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.295087 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.295176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.295134 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.295176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.295152 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh5f9\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-kube-api-access-wh5f9\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.295176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.295167 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.295426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.295182 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.295426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.295196 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.295426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.295210 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.295426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.295223 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.339927 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.339892 5627 generic.go:356] "Generic (PLEG): container finished" podID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerID="9cdda0f05c77105a42ac39a01eb692a892ebd4e0d62eb9f7d07ae887f292f727" exitCode=0 Oct 30 19:33:18.340313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.339940 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" event={"ID":"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317","Type":"ContainerDied","Data":"9cdda0f05c77105a42ac39a01eb692a892ebd4e0d62eb9f7d07ae887f292f727"} Oct 30 19:33:18.340313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.340061 5627 scope.go:122] "RemoveContainer" containerID="a0a9770176072a6a9164006fb1330271e8f90bb901f43cbe621d8803b2950651" Oct 30 19:33:18.344317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.344293 5627 generic.go:356] "Generic (PLEG): container finished" podID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerID="c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea" exitCode=0 Oct 30 19:33:18.344421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.344394 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" event={"ID":"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2","Type":"ContainerDied","Data":"c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea"} Oct 30 19:33:18.344473 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.344430 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" event={"ID":"c2e5a633-4a42-4a8d-9eae-c7805c2cdab2","Type":"ContainerDied","Data":"41b8b37ef30b9f9a2a41ef81dc383eead99a78d260f128d706114ef80a32b31b"} Oct 30 19:33:18.344510 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.344487 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6" Oct 30 19:33:18.346577 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.346556 5627 generic.go:356] "Generic (PLEG): container finished" podID="093f3b61-0d77-473c-be63-9b96facf3b61" containerID="b4a355e5d99834716773729fe8d02336f1c98baf0677b8ecb44f2ac896d23dff" exitCode=0 Oct 30 19:33:18.346737 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.346632 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" event={"ID":"093f3b61-0d77-473c-be63-9b96facf3b61","Type":"ContainerDied","Data":"b4a355e5d99834716773729fe8d02336f1c98baf0677b8ecb44f2ac896d23dff"} Oct 30 19:33:18.359340 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.359246 5627 scope.go:122] "RemoveContainer" containerID="c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea" Oct 30 19:33:18.378136 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.378105 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6"] Oct 30 19:33:18.389610 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.389585 5627 scope.go:122] "RemoveContainer" containerID="c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea" Oct 30 19:33:18.390007 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:18.389983 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea\": container with ID starting with c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea not found: ID does not exist" containerID="c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea" Oct 30 19:33:18.390093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.390018 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea"} err="failed to get container status \"c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea\": rpc error: code = NotFound desc = could not find container \"c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea\": container with ID starting with c0bd8617a4ea8c852d9db29396857a7e3c87102d2d6dafcc9fed8639665c1aea not found: ID does not exist" Oct 30 19:33:18.393037 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.393010 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-dxww6"] Oct 30 19:33:18.432436 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.432401 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:33:18.464347 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.464315 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:33:18.498189 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498163 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-certs\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.498374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498200 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-certs\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.498374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498238 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-envoy\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.498374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498281 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-istio-token\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-istio-token\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.498374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498373 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/093f3b61-0d77-473c-be63-9b96facf3b61-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/093f3b61-0d77-473c-be63-9b96facf3b61-istiod-ca-cert\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.498606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498403 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-kube-api-access-67pn9\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-kube-api-access-67pn9\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.498606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498431 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/093f3b61-0d77-473c-be63-9b96facf3b61-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/093f3b61-0d77-473c-be63-9b96facf3b61-istio-podinfo\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.498606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498477 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-socket\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.498606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498524 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-socket\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.498606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498596 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-credential-socket\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.498876 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498622 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-data\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.498876 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498477 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-certs" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.498876 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498690 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-credential-socket\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.498876 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498801 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-kube-api-access-f6zm5\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-kube-api-access-f6zm5\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.498876 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498833 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/093f3b61-0d77-473c-be63-9b96facf3b61-istiod-ca-cert" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:18.498876 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498852 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-token\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-token\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.498876 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.498870 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-credential-socket" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.499230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.499075 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-socket" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.499299 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.499262 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-credential-socket" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.499408 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.499389 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-data" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.499470 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.499433 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istiod-ca-cert\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.499527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.499500 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-envoy\") pod \"093f3b61-0d77-473c-be63-9b96facf3b61\" (UID: \"093f3b61-0d77-473c-be63-9b96facf3b61\") " Oct 30 19:33:18.499622 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.499582 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-podinfo\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.499679 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.499631 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-data\") pod \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\" (UID: \"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317\") " Oct 30 19:33:18.500097 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.499826 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-socket" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.500097 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500007 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-data" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.500290 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500267 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istiod-ca-cert" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:18.500455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500432 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.500527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500460 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.500527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500475 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.500527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500489 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.500527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500503 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/093f3b61-0d77-473c-be63-9b96facf3b61-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.500527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500516 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.500830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500530 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.500830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500544 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.500830 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.500557 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.501111 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.501084 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-certs" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.502227 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.502201 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-istio-token" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:18.502499 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.502476 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-envoy" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.503509 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.503137 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-token" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:18.503509 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.503304 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-kube-api-access-f6zm5" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "kube-api-access-f6zm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:18.505026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.505000 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-kube-api-access-67pn9" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "kube-api-access-67pn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:18.506347 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.506311 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/093f3b61-0d77-473c-be63-9b96facf3b61-istio-podinfo" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:18.506578 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.506553 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-envoy" pod "093f3b61-0d77-473c-be63-9b96facf3b61" (UID: "093f3b61-0d77-473c-be63-9b96facf3b61"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:18.506673 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.506604 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-podinfo" pod "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" (UID: "14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:18.602271 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602197 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.602271 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602234 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.602271 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602248 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.602271 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602263 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.602591 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602277 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67pn9\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-kube-api-access-67pn9\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.602591 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602293 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/093f3b61-0d77-473c-be63-9b96facf3b61-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.602591 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602307 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6zm5\" (UniqueName: \"kubernetes.io/projected/093f3b61-0d77-473c-be63-9b96facf3b61-kube-api-access-f6zm5\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.602591 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602319 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.602591 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.602335 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/093f3b61-0d77-473c-be63-9b96facf3b61-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:18.629435 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.629405 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:18.915477 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:18.915443 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:19.140728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.140684 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn"] Oct 30 19:33:19.159667 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.159635 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4"] Oct 30 19:33:19.269434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.269407 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:19.353429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.353207 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" Oct 30 19:33:19.353429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.353219 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv" event={"ID":"093f3b61-0d77-473c-be63-9b96facf3b61","Type":"ContainerDied","Data":"f1eb1ed389dcaa079e105dc78cd6278a90ee4b0576f78dd0868bc7344cbd8013"} Oct 30 19:33:19.353429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.353333 5627 scope.go:122] "RemoveContainer" containerID="b4a355e5d99834716773729fe8d02336f1c98baf0677b8ecb44f2ac896d23dff" Oct 30 19:33:19.354986 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.354883 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" Oct 30 19:33:19.354986 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.354913 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m" event={"ID":"14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317","Type":"ContainerDied","Data":"72eb99f262a41427baef1f5336b2f4ac1801bf5f2fb72393fe599a9b87aebe27"} Oct 30 19:33:19.364647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.364630 5627 scope.go:122] "RemoveContainer" containerID="9cdda0f05c77105a42ac39a01eb692a892ebd4e0d62eb9f7d07ae887f292f727" Oct 30 19:33:19.384381 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.384354 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv"] Oct 30 19:33:19.390083 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.390065 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-vmbvv"] Oct 30 19:33:19.397551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.397533 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" path="/var/lib/kubelet/pods/093f3b61-0d77-473c-be63-9b96facf3b61/volumes" Oct 30 19:33:19.397999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.397985 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" path="/var/lib/kubelet/pods/c2e5a633-4a42-4a8d-9eae-c7805c2cdab2/volumes" Oct 30 19:33:19.403310 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.403270 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m"] Oct 30 19:33:19.409835 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.409812 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-jx28m"] Oct 30 19:33:19.811985 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:19.811945 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:21.070312 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.070278 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:21.355465 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.355386 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:21.397270 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.397241 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" path="/var/lib/kubelet/pods/14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317/volumes" Oct 30 19:33:21.568390 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.568359 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:21.703822 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.703742 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc"] Oct 30 19:33:21.704042 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.704011 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" containerID="cri-o://4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4" gracePeriod=30 Oct 30 19:33:21.711028 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.711004 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k"] Oct 30 19:33:21.711202 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.711183 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" containerID="cri-o://261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9" gracePeriod=30 Oct 30 19:33:21.712771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.712748 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb"] Oct 30 19:33:21.712923 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.712904 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" containerID="cri-o://4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd" gracePeriod=30 Oct 30 19:33:21.963048 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:21.962964 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:22.211953 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:22.211920 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:23.193429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.193388 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:23.664086 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.664058 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:23.773315 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.773260 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:33:23.843811 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.843769 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:23.865000 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.864975 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1668db65-90fe-4e2f-9ae9-70c50a2c788a-kube-api-access-5bhfq\" (UniqueName: \"kubernetes.io/projected/1668db65-90fe-4e2f-9ae9-70c50a2c788a-kube-api-access-5bhfq\") pod \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\" (UID: \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\") " Oct 30 19:33:23.865229 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.865209 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1668db65-90fe-4e2f-9ae9-70c50a2c788a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1668db65-90fe-4e2f-9ae9-70c50a2c788a-plugins-config-volume\") pod \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\" (UID: \"1668db65-90fe-4e2f-9ae9-70c50a2c788a\") " Oct 30 19:33:23.865626 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.865601 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1668db65-90fe-4e2f-9ae9-70c50a2c788a-plugins-config-volume" pod "1668db65-90fe-4e2f-9ae9-70c50a2c788a" (UID: "1668db65-90fe-4e2f-9ae9-70c50a2c788a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:23.867496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.867474 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1668db65-90fe-4e2f-9ae9-70c50a2c788a-kube-api-access-5bhfq" pod "1668db65-90fe-4e2f-9ae9-70c50a2c788a" (UID: "1668db65-90fe-4e2f-9ae9-70c50a2c788a"). InnerVolumeSpecName "kube-api-access-5bhfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:23.953261 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.953228 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf"] Oct 30 19:33:23.953661 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.953633 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" containerID="cri-o://631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242" gracePeriod=30 Oct 30 19:33:23.958508 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.958478 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x"] Oct 30 19:33:23.958742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.958682 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" containerID="cri-o://690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0" gracePeriod=30 Oct 30 19:33:23.966785 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.966758 5627 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1668db65-90fe-4e2f-9ae9-70c50a2c788a-plugins-config-volume\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:23.966867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.966787 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bhfq\" (UniqueName: \"kubernetes.io/projected/1668db65-90fe-4e2f-9ae9-70c50a2c788a-kube-api-access-5bhfq\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:23.987923 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.987893 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq"] Oct 30 19:33:23.988137 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:23.988114 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" containerID="cri-o://4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef" gracePeriod=30 Oct 30 19:33:24.016951 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.016920 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:24.327543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.327302 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:24.389017 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.388982 5627 generic.go:356] "Generic (PLEG): container finished" podID="1668db65-90fe-4e2f-9ae9-70c50a2c788a" containerID="fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43" exitCode=0 Oct 30 19:33:24.389209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.389033 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" Oct 30 19:33:24.389209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.389058 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" event={"ID":"1668db65-90fe-4e2f-9ae9-70c50a2c788a","Type":"ContainerDied","Data":"fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43"} Oct 30 19:33:24.389209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.389109 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv" event={"ID":"1668db65-90fe-4e2f-9ae9-70c50a2c788a","Type":"ContainerDied","Data":"c256c204f7431b598c778cce343203131682f9656b1cdbc8ce950c80322dfde1"} Oct 30 19:33:24.389209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.389130 5627 scope.go:122] "RemoveContainer" containerID="fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43" Oct 30 19:33:24.399275 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.399204 5627 scope.go:122] "RemoveContainer" containerID="fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43" Oct 30 19:33:24.399650 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:24.399630 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43\": container with ID starting with fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43 not found: ID does not exist" containerID="fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43" Oct 30 19:33:24.399763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.399659 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43"} err="failed to get container status \"fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43\": rpc error: code = NotFound desc = could not find container \"fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43\": container with ID starting with fdac2960f78fbf4c4c66c3d5e08c0846aa1db96c1ec21da26b06c55f66d97d43 not found: ID does not exist" Oct 30 19:33:24.424014 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.423958 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv"] Oct 30 19:33:24.426598 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.426575 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-6cf459-7f6bfb976c-2w6dv"] Oct 30 19:33:24.526630 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.526588 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk"] Oct 30 19:33:24.661918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.661842 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:24.818346 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.818290 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:24.959588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.959511 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:33:24.959588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:24.959566 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:33:25.398178 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:25.398147 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1668db65-90fe-4e2f-9ae9-70c50a2c788a" path="/var/lib/kubelet/pods/1668db65-90fe-4e2f-9ae9-70c50a2c788a/volumes" Oct 30 19:33:25.976867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:25.976829 5627 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:33:25.977256 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:25.976896 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:33:26.256379 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:26.256348 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:26.994562 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:26.994536 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:33:27.009569 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.009544 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:33:27.021930 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.021907 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:33:27.105516 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105476 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-data\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.105656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105523 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-envoy\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.105656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105560 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-socket\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.105656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105581 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/77a4921d-b562-4871-b94c-ae5896d7a997-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77a4921d-b562-4871-b94c-ae5896d7a997-istio-podinfo\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.105656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105623 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-credential-socket\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.105656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105639 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-certs\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.105968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105815 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-data\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.105968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105807 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-data" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.105968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105853 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-token\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-token\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.105968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105895 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istiod-ca-cert\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.105968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105908 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-credential-socket" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.105968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105916 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-socket" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.105968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.105925 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-envoy\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106018 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-kube-api-access-nckfs\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-kube-api-access-nckfs\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106048 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-certs" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106056 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-credential-socket\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106083 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-data" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106089 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-socket\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106145 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77a4921d-b562-4871-b94c-ae5896d7a997-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77a4921d-b562-4871-b94c-ae5896d7a997-istiod-ca-cert\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106175 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-istio-token\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-istio-token\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106265 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-data\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.106318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106300 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-kube-api-access-48h6v\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-kube-api-access-48h6v\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106336 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istiod-ca-cert" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106351 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-token\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-token\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106382 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-credential-socket" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106402 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-socket\") pod \"77a4921d-b562-4871-b94c-ae5896d7a997\" (UID: \"77a4921d-b562-4871-b94c-ae5896d7a997\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106431 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-envoy\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106461 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-credential-socket\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106493 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-kube-api-access-nxmg7\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-kube-api-access-nxmg7\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106494 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77a4921d-b562-4871-b94c-ae5896d7a997-istiod-ca-cert" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106523 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-certs\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106560 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-certs\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106582 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-data" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106611 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebf02d4a-73a3-4e29-9504-64c516a4869a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf02d4a-73a3-4e29-9504-64c516a4869a-istiod-ca-cert\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106649 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-podinfo\") pod \"ebf02d4a-73a3-4e29-9504-64c516a4869a\" (UID: \"ebf02d4a-73a3-4e29-9504-64c516a4869a\") " Oct 30 19:33:27.106778 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106738 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-podinfo\") pod \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\" (UID: \"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db\") " Oct 30 19:33:27.107455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.106806 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-socket" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.107455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107047 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-credential-socket" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.107455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107397 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107420 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107436 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107451 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107466 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107481 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107496 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107509 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77a4921d-b562-4871-b94c-ae5896d7a997-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107523 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107537 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107551 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.107948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.107937 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-certs" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.108364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.108123 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-certs" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.108813 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.108451 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-socket" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.108969 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.108936 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebf02d4a-73a3-4e29-9504-64c516a4869a-istiod-ca-cert" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:27.109069 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.109049 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-kube-api-access-nckfs" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "kube-api-access-nckfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:27.109157 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.109132 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-istio-token" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:27.109217 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.109202 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-envoy" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.109537 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.109509 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-podinfo" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:27.110334 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.110293 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77a4921d-b562-4871-b94c-ae5896d7a997-istio-podinfo" pod "77a4921d-b562-4871-b94c-ae5896d7a997" (UID: "77a4921d-b562-4871-b94c-ae5896d7a997"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:27.110463 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.110394 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-envoy" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.111020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.110997 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-token" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:27.111188 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.111161 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-envoy" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:27.111324 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.111307 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-kube-api-access-48h6v" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "kube-api-access-48h6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:27.111496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.111472 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-podinfo" pod "ebf02d4a-73a3-4e29-9504-64c516a4869a" (UID: "ebf02d4a-73a3-4e29-9504-64c516a4869a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:27.111766 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.111752 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-kube-api-access-nxmg7" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "kube-api-access-nxmg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:27.111986 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.111961 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-token" pod "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" (UID: "9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:27.208619 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208593 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208619 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208617 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208627 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nckfs\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-kube-api-access-nckfs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208638 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77a4921d-b562-4871-b94c-ae5896d7a997-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208646 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48h6v\" (UniqueName: \"kubernetes.io/projected/ebf02d4a-73a3-4e29-9504-64c516a4869a-kube-api-access-48h6v\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208654 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208663 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208671 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208678 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxmg7\" (UniqueName: \"kubernetes.io/projected/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-kube-api-access-nxmg7\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208686 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208694 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf02d4a-73a3-4e29-9504-64c516a4869a-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208731 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf02d4a-73a3-4e29-9504-64c516a4869a-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208743 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf02d4a-73a3-4e29-9504-64c516a4869a-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208753 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208761 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77a4921d-b562-4871-b94c-ae5896d7a997-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.208814 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.208771 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77a4921d-b562-4871-b94c-ae5896d7a997-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:27.412638 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.412602 5627 generic.go:356] "Generic (PLEG): container finished" podID="77a4921d-b562-4871-b94c-ae5896d7a997" containerID="4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4" exitCode=0 Oct 30 19:33:27.412825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.412664 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" Oct 30 19:33:27.412825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.412761 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" event={"ID":"77a4921d-b562-4871-b94c-ae5896d7a997","Type":"ContainerDied","Data":"4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4"} Oct 30 19:33:27.412825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.412815 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc" event={"ID":"77a4921d-b562-4871-b94c-ae5896d7a997","Type":"ContainerDied","Data":"bf9bdbc5d608144bb9cc74e741f8ffdfa9469d9ff531d39b9efde3462eb859ee"} Oct 30 19:33:27.413006 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.412845 5627 scope.go:122] "RemoveContainer" containerID="4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4" Oct 30 19:33:27.415123 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.415101 5627 generic.go:356] "Generic (PLEG): container finished" podID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerID="261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9" exitCode=0 Oct 30 19:33:27.415272 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.415157 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" Oct 30 19:33:27.415272 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.415195 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" event={"ID":"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db","Type":"ContainerDied","Data":"261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9"} Oct 30 19:33:27.415272 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.415236 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k" event={"ID":"9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db","Type":"ContainerDied","Data":"0d97d08704da6060ffd9a2ed84e45f2fffbaa5e1e9b9b82b19d791b7df814e21"} Oct 30 19:33:27.417194 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.417170 5627 generic.go:356] "Generic (PLEG): container finished" podID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerID="4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd" exitCode=0 Oct 30 19:33:27.417310 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.417220 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" Oct 30 19:33:27.417310 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.417268 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" event={"ID":"ebf02d4a-73a3-4e29-9504-64c516a4869a","Type":"ContainerDied","Data":"4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd"} Oct 30 19:33:27.417421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.417345 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb" event={"ID":"ebf02d4a-73a3-4e29-9504-64c516a4869a","Type":"ContainerDied","Data":"7799ab56cd23d5310e5e0f2b847800602909139b7d5cbcac25e6a5caf273c343"} Oct 30 19:33:27.425674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.425659 5627 scope.go:122] "RemoveContainer" containerID="22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7" Oct 30 19:33:27.435349 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.435332 5627 scope.go:122] "RemoveContainer" containerID="4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4" Oct 30 19:33:27.435658 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:27.435632 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4\": container with ID starting with 4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4 not found: ID does not exist" containerID="4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4" Oct 30 19:33:27.435764 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.435669 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4"} err="failed to get container status \"4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4\": rpc error: code = NotFound desc = could not find container \"4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4\": container with ID starting with 4a21ffba1be839e30fa5cee05df3491ed0c42013d5199d640ca8c83255b214d4 not found: ID does not exist" Oct 30 19:33:27.435764 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.435691 5627 scope.go:122] "RemoveContainer" containerID="22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7" Oct 30 19:33:27.435996 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:27.435976 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7\": container with ID starting with 22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7 not found: ID does not exist" containerID="22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7" Oct 30 19:33:27.436041 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.436005 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7"} err="failed to get container status \"22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7\": rpc error: code = NotFound desc = could not find container \"22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7\": container with ID starting with 22c2fab97e32a8c7a3afe47fbbf0e39a1aa0073772f5fb516fdcd2eff23bc4d7 not found: ID does not exist" Oct 30 19:33:27.436041 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.436027 5627 scope.go:122] "RemoveContainer" containerID="261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9" Oct 30 19:33:27.444421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.444403 5627 scope.go:122] "RemoveContainer" containerID="261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9" Oct 30 19:33:27.444684 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:27.444664 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9\": container with ID starting with 261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9 not found: ID does not exist" containerID="261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9" Oct 30 19:33:27.444802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.444687 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9"} err="failed to get container status \"261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9\": rpc error: code = NotFound desc = could not find container \"261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9\": container with ID starting with 261ad0c6620b06c945f959584b51d1077db2dd9c775376d034fd89abb3f61ae9 not found: ID does not exist" Oct 30 19:33:27.444802 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.444733 5627 scope.go:122] "RemoveContainer" containerID="4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd" Oct 30 19:33:27.451415 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.451392 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb"] Oct 30 19:33:27.453279 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.453258 5627 scope.go:122] "RemoveContainer" containerID="4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd" Oct 30 19:33:27.453569 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:27.453542 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd\": container with ID starting with 4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd not found: ID does not exist" containerID="4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd" Oct 30 19:33:27.453634 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.453573 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd"} err="failed to get container status \"4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd\": rpc error: code = NotFound desc = could not find container \"4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd\": container with ID starting with 4ba9e54d902a4531d8206d5cd3bfda134de194174a8f394a123e99ea483054cd not found: ID does not exist" Oct 30 19:33:27.453689 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.453675 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-92tlb"] Oct 30 19:33:27.466789 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.466765 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc"] Oct 30 19:33:27.468813 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.468795 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-dhnpc"] Oct 30 19:33:27.481940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.481914 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k"] Oct 30 19:33:27.484638 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:27.484609 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-vsl5k"] Oct 30 19:33:28.364639 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:28.364604 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:28.491648 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:28.491609 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:29.231659 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.231596 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:33:29.265150 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.265127 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:33:29.293335 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.293307 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:33:29.334645 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334620 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-kube-api-access-bdcvx\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-kube-api-access-bdcvx\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.334836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334671 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-credential-socket\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.334836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334693 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-podinfo\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.334836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334777 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-envoy\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.334836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334799 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-data\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.334836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334821 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-certs\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.335109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334859 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-podinfo\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.335109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334884 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-envoy\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.335109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334923 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-data\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.335109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334966 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-credential-socket" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.335109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.334990 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-token\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-token\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.335109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335028 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-socket\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.335109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335062 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-kube-api-access-vfnhd\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-kube-api-access-vfnhd\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.335109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335103 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-certs\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.335490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335200 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca751225-aab9-4454-ab45-f9ace259a4ff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751225-aab9-4454-ab45-f9ace259a4ff-istiod-ca-cert\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.335490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335300 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-token\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-token\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.335490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335326 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-certs" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.335490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335335 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-credential-socket\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.335490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335407 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-socket\") pod \"ca751225-aab9-4454-ab45-f9ace259a4ff\" (UID: \"ca751225-aab9-4454-ab45-f9ace259a4ff\") " Oct 30 19:33:29.335744 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335491 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4989abce-250d-49ac-9fbb-b45d09a852a2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4989abce-250d-49ac-9fbb-b45d09a852a2-istiod-ca-cert\") pod \"4989abce-250d-49ac-9fbb-b45d09a852a2\" (UID: \"4989abce-250d-49ac-9fbb-b45d09a852a2\") " Oct 30 19:33:29.335744 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335519 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-credential-socket" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.335744 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.335723 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-socket" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336005 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4989abce-250d-49ac-9fbb-b45d09a852a2-istiod-ca-cert" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336092 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-certs" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336197 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-data" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336238 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336415 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336436 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336452 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336417 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-data" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336434 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-socket" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.336771 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.336692 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca751225-aab9-4454-ab45-f9ace259a4ff-istiod-ca-cert" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:29.337953 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.337914 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-envoy" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.339538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.339502 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-kube-api-access-vfnhd" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "kube-api-access-vfnhd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:29.340563 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.340529 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-token" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:29.341131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.341102 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-envoy" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.341566 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.341538 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-podinfo" pod "ca751225-aab9-4454-ab45-f9ace259a4ff" (UID: "ca751225-aab9-4454-ab45-f9ace259a4ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:29.341850 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.341832 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-token" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:29.343051 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.343030 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-kube-api-access-bdcvx" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "kube-api-access-bdcvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:29.344931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.344911 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-podinfo" pod "4989abce-250d-49ac-9fbb-b45d09a852a2" (UID: "4989abce-250d-49ac-9fbb-b45d09a852a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:29.399536 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.399496 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" path="/var/lib/kubelet/pods/77a4921d-b562-4871-b94c-ae5896d7a997/volumes" Oct 30 19:33:29.400210 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.400188 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" path="/var/lib/kubelet/pods/9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db/volumes" Oct 30 19:33:29.400716 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.400684 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" path="/var/lib/kubelet/pods/ebf02d4a-73a3-4e29-9504-64c516a4869a/volumes" Oct 30 19:33:29.435432 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.435405 5627 generic.go:356] "Generic (PLEG): container finished" podID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerID="690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0" exitCode=0 Oct 30 19:33:29.435560 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.435466 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" Oct 30 19:33:29.435560 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.435488 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" event={"ID":"49657d5c-b0b6-4abd-9a6f-dc8b3071e212","Type":"ContainerDied","Data":"690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0"} Oct 30 19:33:29.435689 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.435564 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x" event={"ID":"49657d5c-b0b6-4abd-9a6f-dc8b3071e212","Type":"ContainerDied","Data":"8d4c381ce9b9e36f1f871427fc8dc29efc07b181dacc1f0817cd1fec2d6c5942"} Oct 30 19:33:29.435689 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.435594 5627 scope.go:122] "RemoveContainer" containerID="690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0" Oct 30 19:33:29.436891 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.436869 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-kube-api-access-hbbxl\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-kube-api-access-hbbxl\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.436961 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.436907 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-token\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-token\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.437021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.436955 5627 generic.go:356] "Generic (PLEG): container finished" podID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerID="631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242" exitCode=0 Oct 30 19:33:29.437021 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.436973 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istiod-ca-cert\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.437127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437022 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-credential-socket\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.437127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437038 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" Oct 30 19:33:29.437127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437056 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-podinfo\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.437127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437113 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-data\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.437321 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437124 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" event={"ID":"ca751225-aab9-4454-ab45-f9ace259a4ff","Type":"ContainerDied","Data":"631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242"} Oct 30 19:33:29.437321 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437154 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-certs\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.437321 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437166 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf" event={"ID":"ca751225-aab9-4454-ab45-f9ace259a4ff","Type":"ContainerDied","Data":"9c205183dd546b2ae8b748c1eb71af9a1fed247a2be5fb3349587a4802a93f27"} Oct 30 19:33:29.437485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437334 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-envoy\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.437485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437416 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-socket\") pod \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\" (UID: \"49657d5c-b0b6-4abd-9a6f-dc8b3071e212\") " Oct 30 19:33:29.437859 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437836 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.437952 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437868 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.437952 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437887 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.437952 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437901 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.437952 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437916 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.437952 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437929 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.437952 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437943 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437957 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfnhd\" (UniqueName: \"kubernetes.io/projected/ca751225-aab9-4454-ab45-f9ace259a4ff-kube-api-access-vfnhd\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437971 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437967 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-certs" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437986 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751225-aab9-4454-ab45-f9ace259a4ff-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.437999 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.438015 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751225-aab9-4454-ab45-f9ace259a4ff-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.438028 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4989abce-250d-49ac-9fbb-b45d09a852a2-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.438043 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdcvx\" (UniqueName: \"kubernetes.io/projected/4989abce-250d-49ac-9fbb-b45d09a852a2-kube-api-access-bdcvx\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.438247 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.438155 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-data" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.438690 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.438248 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-socket" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.438690 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.438458 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istiod-ca-cert" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:29.438823 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.438724 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-credential-socket" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.439011 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.438987 5627 generic.go:356] "Generic (PLEG): container finished" podID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerID="4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef" exitCode=0 Oct 30 19:33:29.439675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.439160 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" Oct 30 19:33:29.439675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.439214 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" event={"ID":"4989abce-250d-49ac-9fbb-b45d09a852a2","Type":"ContainerDied","Data":"4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef"} Oct 30 19:33:29.439675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.439489 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq" event={"ID":"4989abce-250d-49ac-9fbb-b45d09a852a2","Type":"ContainerDied","Data":"29926ac4e7376a90c3c773212647b387f293447ffd73b677fcfc1c8fef2cbdb9"} Oct 30 19:33:29.441661 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.441636 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-kube-api-access-hbbxl" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "kube-api-access-hbbxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:29.442061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.442029 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-envoy" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:29.442659 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.442635 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-token" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:29.442980 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.442954 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-podinfo" pod "49657d5c-b0b6-4abd-9a6f-dc8b3071e212" (UID: "49657d5c-b0b6-4abd-9a6f-dc8b3071e212"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:29.448201 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.448173 5627 scope.go:122] "RemoveContainer" containerID="690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0" Oct 30 19:33:29.448519 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:29.448484 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0\": container with ID starting with 690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0 not found: ID does not exist" containerID="690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0" Oct 30 19:33:29.448581 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.448531 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0"} err="failed to get container status \"690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0\": rpc error: code = NotFound desc = could not find container \"690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0\": container with ID starting with 690c4f99f52eba2bacbe61ec4540ca5499e7b07e916e0bb64a77ae6206fa97b0 not found: ID does not exist" Oct 30 19:33:29.448581 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.448557 5627 scope.go:122] "RemoveContainer" containerID="631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242" Oct 30 19:33:29.457191 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.457175 5627 scope.go:122] "RemoveContainer" containerID="631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242" Oct 30 19:33:29.457491 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:29.457473 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242\": container with ID starting with 631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242 not found: ID does not exist" containerID="631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242" Oct 30 19:33:29.457543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.457497 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242"} err="failed to get container status \"631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242\": rpc error: code = NotFound desc = could not find container \"631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242\": container with ID starting with 631142b07a2195a55ce2db890267b07fe0af8fff415c957b5f0c27aa46d70242 not found: ID does not exist" Oct 30 19:33:29.457543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.457522 5627 scope.go:122] "RemoveContainer" containerID="4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef" Oct 30 19:33:29.466054 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.466038 5627 scope.go:122] "RemoveContainer" containerID="4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef" Oct 30 19:33:29.466316 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:29.466301 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef\": container with ID starting with 4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef not found: ID does not exist" containerID="4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef" Oct 30 19:33:29.466373 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.466321 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef"} err="failed to get container status \"4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef\": rpc error: code = NotFound desc = could not find container \"4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef\": container with ID starting with 4212639e57e8ed7f97ee2dd5ae1ccd7d0b311f89f2990e7fd6a0a4150bd8c0ef not found: ID does not exist" Oct 30 19:33:29.470564 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.470542 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf"] Oct 30 19:33:29.476593 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.476572 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-72zhf"] Oct 30 19:33:29.489817 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.489788 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq"] Oct 30 19:33:29.496568 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.496548 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-fc9kq"] Oct 30 19:33:29.527071 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.527031 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" podUID="8e3f421b-8c94-4018-92c1-f5655816d945" containerName="payload-processing" containerID="cri-o://2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862" gracePeriod=25 Oct 30 19:33:29.539303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539274 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbbxl\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-kube-api-access-hbbxl\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.539303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539301 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.539517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539313 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.539517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539322 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.539517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539330 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.539517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539344 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.539517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539354 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.539517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539362 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.539517 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.539372 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49657d5c-b0b6-4abd-9a6f-dc8b3071e212-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:29.759419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.759393 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x"] Oct 30 19:33:29.766085 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:29.766052 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-s5m6x"] Oct 30 19:33:30.061936 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:30.061857 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:30.197105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:30.197061 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:30.426501 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:30.426403 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:30.557909 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:30.557878 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:30.878096 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:30.878051 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:31.402896 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:31.402858 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" path="/var/lib/kubelet/pods/49657d5c-b0b6-4abd-9a6f-dc8b3071e212/volumes" Oct 30 19:33:31.403364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:31.403344 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" path="/var/lib/kubelet/pods/4989abce-250d-49ac-9fbb-b45d09a852a2/volumes" Oct 30 19:33:31.403817 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:31.403801 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" path="/var/lib/kubelet/pods/ca751225-aab9-4454-ab45-f9ace259a4ff/volumes" Oct 30 19:33:31.602188 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:31.602152 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:34.162943 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.162894 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" containerID="cri-o://3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3" gracePeriod=45 Oct 30 19:33:34.179952 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.179914 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" containerID="cri-o://227ceed5896dbc42feb464dd9c958a10882535acf86ce3578716bce11661112f" gracePeriod=45 Oct 30 19:33:34.471637 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.471615 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:33:34.475002 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.474977 5627 generic.go:356] "Generic (PLEG): container finished" podID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerID="3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3" exitCode=0 Oct 30 19:33:34.475142 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.475018 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" Oct 30 19:33:34.475142 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.475066 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" event={"ID":"2350a8f2-ee45-47dc-aca4-138b0a907df8","Type":"ContainerDied","Data":"3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3"} Oct 30 19:33:34.475142 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.475110 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn" event={"ID":"2350a8f2-ee45-47dc-aca4-138b0a907df8","Type":"ContainerDied","Data":"9bd4728ccac3ceee52828798ae7b9e5ea0ef4e3049aca37702a0a596893558e6"} Oct 30 19:33:34.475142 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.475129 5627 scope.go:122] "RemoveContainer" containerID="3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3" Oct 30 19:33:34.476370 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.476351 5627 generic.go:356] "Generic (PLEG): container finished" podID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerID="227ceed5896dbc42feb464dd9c958a10882535acf86ce3578716bce11661112f" exitCode=0 Oct 30 19:33:34.476474 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.476431 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" event={"ID":"8ea159b5-d465-4a12-83ec-ad86afa91fec","Type":"ContainerDied","Data":"227ceed5896dbc42feb464dd9c958a10882535acf86ce3578716bce11661112f"} Oct 30 19:33:34.476534 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.476478 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" event={"ID":"8ea159b5-d465-4a12-83ec-ad86afa91fec","Type":"ContainerDied","Data":"0f2dabc358460a9799ab6979729f43c3be761a3a647542474fbbd8f4d7df4390"} Oct 30 19:33:34.476534 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.476496 5627 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f2dabc358460a9799ab6979729f43c3be761a3a647542474fbbd8f4d7df4390" Oct 30 19:33:34.478099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.478082 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:33:34.485049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.485028 5627 scope.go:122] "RemoveContainer" containerID="3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3" Oct 30 19:33:34.485300 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:34.485281 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3\": container with ID starting with 3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3 not found: ID does not exist" containerID="3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3" Oct 30 19:33:34.485368 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.485312 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3"} err="failed to get container status \"3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3\": rpc error: code = NotFound desc = could not find container \"3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3\": container with ID starting with 3c933f6daf07848cfeff6ba7b2b7b6f32903fda812a8fd1d1e0ebdaae37b77e3 not found: ID does not exist" Oct 30 19:33:34.595394 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.595362 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-dshm\") pod \"2350a8f2-ee45-47dc-aca4-138b0a907df8\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " Oct 30 19:33:34.595629 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.595611 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-tmp-dir\") pod \"8ea159b5-d465-4a12-83ec-ad86afa91fec\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " Oct 30 19:33:34.595847 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.595830 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-home\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-home\") pod \"2350a8f2-ee45-47dc-aca4-138b0a907df8\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " Oct 30 19:33:34.595982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.595967 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-model-cache\") pod \"8ea159b5-d465-4a12-83ec-ad86afa91fec\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " Oct 30 19:33:34.596111 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596098 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-tmp-dir\") pod \"2350a8f2-ee45-47dc-aca4-138b0a907df8\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " Oct 30 19:33:34.596421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596396 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ea159b5-d465-4a12-83ec-ad86afa91fec-kube-api-access-btgtq\" (UniqueName: \"kubernetes.io/projected/8ea159b5-d465-4a12-83ec-ad86afa91fec-kube-api-access-btgtq\") pod \"8ea159b5-d465-4a12-83ec-ad86afa91fec\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " Oct 30 19:33:34.596549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596456 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2350a8f2-ee45-47dc-aca4-138b0a907df8-tls-certs\" (UniqueName: \"kubernetes.io/secret/2350a8f2-ee45-47dc-aca4-138b0a907df8-tls-certs\") pod \"2350a8f2-ee45-47dc-aca4-138b0a907df8\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " Oct 30 19:33:34.596549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596506 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-home\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-home\") pod \"8ea159b5-d465-4a12-83ec-ad86afa91fec\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " Oct 30 19:33:34.596549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596539 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-model-cache\") pod \"2350a8f2-ee45-47dc-aca4-138b0a907df8\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " Oct 30 19:33:34.596768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596572 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2350a8f2-ee45-47dc-aca4-138b0a907df8-kube-api-access-hm4tf\" (UniqueName: \"kubernetes.io/projected/2350a8f2-ee45-47dc-aca4-138b0a907df8-kube-api-access-hm4tf\") pod \"2350a8f2-ee45-47dc-aca4-138b0a907df8\" (UID: \"2350a8f2-ee45-47dc-aca4-138b0a907df8\") " Oct 30 19:33:34.596768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596646 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-dshm\") pod \"8ea159b5-d465-4a12-83ec-ad86afa91fec\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " Oct 30 19:33:34.596768 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596675 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ea159b5-d465-4a12-83ec-ad86afa91fec-tls-certs\" (UniqueName: \"kubernetes.io/secret/8ea159b5-d465-4a12-83ec-ad86afa91fec-tls-certs\") pod \"8ea159b5-d465-4a12-83ec-ad86afa91fec\" (UID: \"8ea159b5-d465-4a12-83ec-ad86afa91fec\") " Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596007 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-tmp-dir" pod "8ea159b5-d465-4a12-83ec-ad86afa91fec" (UID: "8ea159b5-d465-4a12-83ec-ad86afa91fec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596223 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-model-cache" pod "8ea159b5-d465-4a12-83ec-ad86afa91fec" (UID: "8ea159b5-d465-4a12-83ec-ad86afa91fec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596235 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-home" pod "2350a8f2-ee45-47dc-aca4-138b0a907df8" (UID: "2350a8f2-ee45-47dc-aca4-138b0a907df8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596388 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-tmp-dir" pod "2350a8f2-ee45-47dc-aca4-138b0a907df8" (UID: "2350a8f2-ee45-47dc-aca4-138b0a907df8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596840 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-model-cache" pod "2350a8f2-ee45-47dc-aca4-138b0a907df8" (UID: "2350a8f2-ee45-47dc-aca4-138b0a907df8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.596984 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-home" pod "8ea159b5-d465-4a12-83ec-ad86afa91fec" (UID: "8ea159b5-d465-4a12-83ec-ad86afa91fec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.597330 5627 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-tmp-dir\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.597362 5627 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-home\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.597377 5627 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-model-cache\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.597390 5627 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-tmp-dir\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.597404 5627 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-home\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.597479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.597432 5627 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-model-cache\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.598399 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.598376 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-dshm" pod "2350a8f2-ee45-47dc-aca4-138b0a907df8" (UID: "2350a8f2-ee45-47dc-aca4-138b0a907df8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:34.599061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.599038 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2350a8f2-ee45-47dc-aca4-138b0a907df8-tls-certs" pod "2350a8f2-ee45-47dc-aca4-138b0a907df8" (UID: "2350a8f2-ee45-47dc-aca4-138b0a907df8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:34.599944 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.599921 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ea159b5-d465-4a12-83ec-ad86afa91fec-tls-certs" pod "8ea159b5-d465-4a12-83ec-ad86afa91fec" (UID: "8ea159b5-d465-4a12-83ec-ad86afa91fec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:34.600028 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.599964 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea159b5-d465-4a12-83ec-ad86afa91fec-kube-api-access-btgtq" pod "8ea159b5-d465-4a12-83ec-ad86afa91fec" (UID: "8ea159b5-d465-4a12-83ec-ad86afa91fec"). InnerVolumeSpecName "kube-api-access-btgtq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:34.601595 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.601575 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-dshm" pod "8ea159b5-d465-4a12-83ec-ad86afa91fec" (UID: "8ea159b5-d465-4a12-83ec-ad86afa91fec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:34.602091 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.602077 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2350a8f2-ee45-47dc-aca4-138b0a907df8-kube-api-access-hm4tf" pod "2350a8f2-ee45-47dc-aca4-138b0a907df8" (UID: "2350a8f2-ee45-47dc-aca4-138b0a907df8"). InnerVolumeSpecName "kube-api-access-hm4tf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:34.698421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.698350 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btgtq\" (UniqueName: \"kubernetes.io/projected/8ea159b5-d465-4a12-83ec-ad86afa91fec-kube-api-access-btgtq\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.698421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.698377 5627 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2350a8f2-ee45-47dc-aca4-138b0a907df8-tls-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.698421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.698387 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm4tf\" (UniqueName: \"kubernetes.io/projected/2350a8f2-ee45-47dc-aca4-138b0a907df8-kube-api-access-hm4tf\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.698421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.698396 5627 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ea159b5-d465-4a12-83ec-ad86afa91fec-dshm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.698421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.698404 5627 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ea159b5-d465-4a12-83ec-ad86afa91fec-tls-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.698421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.698411 5627 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2350a8f2-ee45-47dc-aca4-138b0a907df8-dshm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:34.893575 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.893513 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn"] Oct 30 19:33:34.896586 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:34.896415 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/distinct-2-w4-3bc085-kserve-97dbcf8c5-94qvn"] Oct 30 19:33:35.397973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:35.397937 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" path="/var/lib/kubelet/pods/2350a8f2-ee45-47dc-aca4-138b0a907df8/volumes" Oct 30 19:33:35.486124 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:35.486053 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4" Oct 30 19:33:35.509171 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:35.509142 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4"] Oct 30 19:33:35.517598 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:35.517568 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/premium-w4-3bc085-kserve-797d79d64f-nr8h4"] Oct 30 19:33:36.825578 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:36.825545 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:37.381558 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:37.381521 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" podUID="8e3f421b-8c94-4018-92c1-f5655816d945" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:33:37.397383 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:37.397354 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" path="/var/lib/kubelet/pods/8ea159b5-d465-4a12-83ec-ad86afa91fec/volumes" Oct 30 19:33:37.620899 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:37.620869 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:39.877098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:39.877072 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2"] Oct 30 19:33:39.877640 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:39.877316 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" podUID="e75aa317-1733-4e9f-af92-163add3ff6ab" containerName="maas-api" containerID="cri-o://82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1" gracePeriod=30 Oct 30 19:33:40.013320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.013285 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:40.183647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.183625 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:33:40.193424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.193398 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:40.280865 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.280831 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-maas-api-tls\") pod \"e75aa317-1733-4e9f-af92-163add3ff6ab\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " Oct 30 19:33:40.281049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.280904 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-metrics-certs\") pod \"e75aa317-1733-4e9f-af92-163add3ff6ab\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " Oct 30 19:33:40.281049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.280953 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e75aa317-1733-4e9f-af92-163add3ff6ab-kube-api-access-mv8ts\" (UniqueName: \"kubernetes.io/projected/e75aa317-1733-4e9f-af92-163add3ff6ab-kube-api-access-mv8ts\") pod \"e75aa317-1733-4e9f-af92-163add3ff6ab\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " Oct 30 19:33:40.281049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.281035 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e75aa317-1733-4e9f-af92-163add3ff6ab-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75aa317-1733-4e9f-af92-163add3ff6ab-openshift-service-ca\") pod \"e75aa317-1733-4e9f-af92-163add3ff6ab\" (UID: \"e75aa317-1733-4e9f-af92-163add3ff6ab\") " Oct 30 19:33:40.281434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.281404 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e75aa317-1733-4e9f-af92-163add3ff6ab-openshift-service-ca" pod "e75aa317-1733-4e9f-af92-163add3ff6ab" (UID: "e75aa317-1733-4e9f-af92-163add3ff6ab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:40.283167 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.283143 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-metrics-certs" pod "e75aa317-1733-4e9f-af92-163add3ff6ab" (UID: "e75aa317-1733-4e9f-af92-163add3ff6ab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:40.283248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.283178 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e75aa317-1733-4e9f-af92-163add3ff6ab-kube-api-access-mv8ts" pod "e75aa317-1733-4e9f-af92-163add3ff6ab" (UID: "e75aa317-1733-4e9f-af92-163add3ff6ab"). InnerVolumeSpecName "kube-api-access-mv8ts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:40.283248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.283208 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-maas-api-tls" pod "e75aa317-1733-4e9f-af92-163add3ff6ab" (UID: "e75aa317-1733-4e9f-af92-163add3ff6ab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:40.349104 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.349065 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:40.382193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.382166 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75aa317-1733-4e9f-af92-163add3ff6ab-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:40.382193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.382191 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:40.382193 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.382201 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75aa317-1733-4e9f-af92-163add3ff6ab-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:40.382401 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.382211 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv8ts\" (UniqueName: \"kubernetes.io/projected/e75aa317-1733-4e9f-af92-163add3ff6ab-kube-api-access-mv8ts\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:40.406224 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:40.406187 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:33:40.514911 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.514877 5627 generic.go:356] "Generic (PLEG): container finished" podID="e75aa317-1733-4e9f-af92-163add3ff6ab" containerID="82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1" exitCode=0 Oct 30 19:33:40.515068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.514924 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" Oct 30 19:33:40.515068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.514954 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" event={"ID":"e75aa317-1733-4e9f-af92-163add3ff6ab","Type":"ContainerDied","Data":"82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1"} Oct 30 19:33:40.515068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.515006 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2" event={"ID":"e75aa317-1733-4e9f-af92-163add3ff6ab","Type":"ContainerDied","Data":"16e87a221beac7d4825ae2b4ed33eb46dab7b138c033b05b1b0c190666ccedf2"} Oct 30 19:33:40.515068 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.515041 5627 scope.go:122] "RemoveContainer" containerID="82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1" Oct 30 19:33:40.524221 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.524206 5627 scope.go:122] "RemoveContainer" containerID="82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1" Oct 30 19:33:40.524512 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:40.524490 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1\": container with ID starting with 82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1 not found: ID does not exist" containerID="82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1" Oct 30 19:33:40.524585 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.524521 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1"} err="failed to get container status \"82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1\": rpc error: code = NotFound desc = could not find container \"82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1\": container with ID starting with 82e4a2e787d0207c16ae4840a1bbb83bec2fbc18879be1b3300912753d8517f1 not found: ID does not exist" Oct 30 19:33:40.539971 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.539949 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2"] Oct 30 19:33:40.545383 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.545364 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3bc085-788cb88f96-9c9k2"] Oct 30 19:33:40.749026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:40.748994 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:41.387332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:41.387284 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:41.397853 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:41.397828 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e75aa317-1733-4e9f-af92-163add3ff6ab" path="/var/lib/kubelet/pods/e75aa317-1733-4e9f-af92-163add3ff6ab/volumes" Oct 30 19:33:44.021619 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:44.021588 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:44.138374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:44.138335 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:45.843878 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:45.843837 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:46.886665 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:46.886632 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:47.191979 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.191888 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:47.321343 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.321307 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4"] Oct 30 19:33:47.321582 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.321557 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" containerID="cri-o://cef44b9b2ea958e9941b52967d54e86a9e71e8a8de7ba26fc12f7dff76fda398" gracePeriod=30 Oct 30 19:33:47.335726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.335410 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r"] Oct 30 19:33:47.335726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.335640 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" containerID="cri-o://753594791f9e05a2158d446691afa3b01b049bfad17ba977e3dd084ef1a50d55" gracePeriod=30 Oct 30 19:33:47.337575 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.337514 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m"] Oct 30 19:33:47.337827 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.337799 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" containerID="cri-o://eb902bf9beeabf084b5221252bcc7559ce007c95417f5dea657333387b5fd414" gracePeriod=30 Oct 30 19:33:47.382018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.381985 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" podUID="8e3f421b-8c94-4018-92c1-f5655816d945" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:33:47.454823 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:47.454733 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:49.776115 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:49.776088 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:33:49.882429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:49.877714 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e3f421b-8c94-4018-92c1-f5655816d945-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3f421b-8c94-4018-92c1-f5655816d945-plugins-config-volume\") pod \"8e3f421b-8c94-4018-92c1-f5655816d945\" (UID: \"8e3f421b-8c94-4018-92c1-f5655816d945\") " Oct 30 19:33:49.882429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:49.877848 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e3f421b-8c94-4018-92c1-f5655816d945-kube-api-access-g4zh5\" (UniqueName: \"kubernetes.io/projected/8e3f421b-8c94-4018-92c1-f5655816d945-kube-api-access-g4zh5\") pod \"8e3f421b-8c94-4018-92c1-f5655816d945\" (UID: \"8e3f421b-8c94-4018-92c1-f5655816d945\") " Oct 30 19:33:49.882429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:49.878104 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3f421b-8c94-4018-92c1-f5655816d945-plugins-config-volume" pod "8e3f421b-8c94-4018-92c1-f5655816d945" (UID: "8e3f421b-8c94-4018-92c1-f5655816d945"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:49.882429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:49.878215 5627 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3f421b-8c94-4018-92c1-f5655816d945-plugins-config-volume\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:49.882429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:49.880155 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3f421b-8c94-4018-92c1-f5655816d945-kube-api-access-g4zh5" pod "8e3f421b-8c94-4018-92c1-f5655816d945" (UID: "8e3f421b-8c94-4018-92c1-f5655816d945"). InnerVolumeSpecName "kube-api-access-g4zh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:49.979109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:49.979087 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4zh5\" (UniqueName: \"kubernetes.io/projected/8e3f421b-8c94-4018-92c1-f5655816d945-kube-api-access-g4zh5\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:50.563631 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.563599 5627 generic.go:356] "Generic (PLEG): container finished" podID="8e3f421b-8c94-4018-92c1-f5655816d945" containerID="2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862" exitCode=0 Oct 30 19:33:50.563845 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.563653 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" Oct 30 19:33:50.563845 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.563664 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" event={"ID":"8e3f421b-8c94-4018-92c1-f5655816d945","Type":"ContainerDied","Data":"2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862"} Oct 30 19:33:50.563845 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.563720 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk" event={"ID":"8e3f421b-8c94-4018-92c1-f5655816d945","Type":"ContainerDied","Data":"9ba9886c8fbc6c1a841aa335cd791db091a52268999b49ef1b44e8a52025ad9c"} Oct 30 19:33:50.563845 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.563742 5627 scope.go:122] "RemoveContainer" containerID="2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862" Oct 30 19:33:50.572777 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.572759 5627 scope.go:122] "RemoveContainer" containerID="2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862" Oct 30 19:33:50.573032 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:33:50.573010 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862\": container with ID starting with 2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862 not found: ID does not exist" containerID="2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862" Oct 30 19:33:50.573100 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.573041 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862"} err="failed to get container status \"2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862\": rpc error: code = NotFound desc = could not find container \"2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862\": container with ID starting with 2c7fbf8839c6bc0dfb4a74e807720adfcb7a236254abbe9154e2e41949cd5862 not found: ID does not exist" Oct 30 19:33:50.587042 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.587017 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk"] Oct 30 19:33:50.594682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:50.594658 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3bc085-db48cdc85-pjnvk"] Oct 30 19:33:51.398710 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:51.398664 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e3f421b-8c94-4018-92c1-f5655816d945" path="/var/lib/kubelet/pods/8e3f421b-8c94-4018-92c1-f5655816d945/volumes" Oct 30 19:33:51.596561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:51.596523 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:33:51.596760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:51.596593 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:33:52.602582 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:52.602540 5627 generic.go:356] "Generic (PLEG): container finished" podID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerID="eb902bf9beeabf084b5221252bcc7559ce007c95417f5dea657333387b5fd414" exitCode=0 Oct 30 19:33:52.603119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:52.602679 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" event={"ID":"e1893e40-3612-4b59-ad5b-ec8144073ea4","Type":"ContainerDied","Data":"eb902bf9beeabf084b5221252bcc7559ce007c95417f5dea657333387b5fd414"} Oct 30 19:33:52.604615 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:52.604448 5627 generic.go:356] "Generic (PLEG): container finished" podID="231c554b-45d4-41d7-9f86-5b2255306400" containerID="753594791f9e05a2158d446691afa3b01b049bfad17ba977e3dd084ef1a50d55" exitCode=0 Oct 30 19:33:52.604615 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:52.604561 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" event={"ID":"231c554b-45d4-41d7-9f86-5b2255306400","Type":"ContainerDied","Data":"753594791f9e05a2158d446691afa3b01b049bfad17ba977e3dd084ef1a50d55"} Oct 30 19:33:52.606362 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:52.606284 5627 generic.go:356] "Generic (PLEG): container finished" podID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerID="cef44b9b2ea958e9941b52967d54e86a9e71e8a8de7ba26fc12f7dff76fda398" exitCode=0 Oct 30 19:33:52.606362 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:52.606312 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" event={"ID":"1c6b620b-3265-4db3-8789-d510ddc149d7","Type":"ContainerDied","Data":"cef44b9b2ea958e9941b52967d54e86a9e71e8a8de7ba26fc12f7dff76fda398"} Oct 30 19:33:52.641452 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:52.641068 5627 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Oct 30 19:33:52.641452 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:52.641129 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Oct 30 19:33:53.007780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.007749 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:33:53.015200 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015165 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-kube-api-access-cnh94\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-kube-api-access-cnh94\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.015323 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015222 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-token\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-token\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.015323 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015287 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-data\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.015484 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015456 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-podinfo\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.015551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015530 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c6b620b-3265-4db3-8789-d510ddc149d7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c6b620b-3265-4db3-8789-d510ddc149d7-istiod-ca-cert\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.015650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015627 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-data" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015841 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-socket\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015866 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c6b620b-3265-4db3-8789-d510ddc149d7-istiod-ca-cert" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015894 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-credential-socket\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015928 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-envoy\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.015965 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-certs\") pod \"1c6b620b-3265-4db3-8789-d510ddc149d7\" (UID: \"1c6b620b-3265-4db3-8789-d510ddc149d7\") " Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.016094 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-socket" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.016288 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-certs" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.016401 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c6b620b-3265-4db3-8789-d510ddc149d7-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.016446 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.016462 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.016477 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.020729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.016460 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-credential-socket" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.107889 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.107819 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-envoy" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.107889 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.107848 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-kube-api-access-cnh94" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "kube-api-access-cnh94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:53.109077 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.109039 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-podinfo" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:53.113114 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.113075 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-token" pod "1c6b620b-3265-4db3-8789-d510ddc149d7" (UID: "1c6b620b-3265-4db3-8789-d510ddc149d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:53.146739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.123471 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.146739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.123504 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.146739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.123520 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnh94\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-kube-api-access-cnh94\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.146739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.123535 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.146739 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.123548 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c6b620b-3265-4db3-8789-d510ddc149d7-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.211593 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.211549 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.228997 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-podinfo\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.229626 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-socket\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.229962 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-socket" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.230244 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-certs\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.230561 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-certs" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.230325 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-kube-api-access-jngrj\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-kube-api-access-jngrj\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.230980 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-envoy\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.231384 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-token\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-token\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.231775 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1893e40-3612-4b59-ad5b-ec8144073ea4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1893e40-3612-4b59-ad5b-ec8144073ea4-istiod-ca-cert\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.232271 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-credential-socket\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.232200 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1893e40-3612-4b59-ad5b-ec8144073ea4-istiod-ca-cert" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:53.232730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.232657 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-credential-socket" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.233505 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.232908 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-data" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.233505 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.232961 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-data\") pod \"e1893e40-3612-4b59-ad5b-ec8144073ea4\" (UID: \"e1893e40-3612-4b59-ad5b-ec8144073ea4\") " Oct 30 19:33:53.233505 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.233428 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1893e40-3612-4b59-ad5b-ec8144073ea4-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.233505 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.233451 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.233505 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.233466 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.233505 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.233480 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.233866 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.233511 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.284078 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.234085 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-podinfo" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:53.284078 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.260728 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-envoy" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.300494 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.300460 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-kube-api-access-jngrj" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "kube-api-access-jngrj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:53.300820 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.300794 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-token" pod "e1893e40-3612-4b59-ad5b-ec8144073ea4" (UID: "e1893e40-3612-4b59-ad5b-ec8144073ea4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:53.310516 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.310495 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378056 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-data\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378126 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/231c554b-45d4-41d7-9f86-5b2255306400-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231c554b-45d4-41d7-9f86-5b2255306400-istio-podinfo\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378183 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-istio-token\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-istio-token\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378222 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-certs\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378264 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-credential-socket\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378331 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/231c554b-45d4-41d7-9f86-5b2255306400-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231c554b-45d4-41d7-9f86-5b2255306400-istiod-ca-cert\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378373 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-kube-api-access-k8kkd\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-kube-api-access-k8kkd\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378407 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-envoy\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378448 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-socket\") pod \"231c554b-45d4-41d7-9f86-5b2255306400\" (UID: \"231c554b-45d4-41d7-9f86-5b2255306400\") " Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378831 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jngrj\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-kube-api-access-jngrj\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378850 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378865 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.378880 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1893e40-3612-4b59-ad5b-ec8144073ea4-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.386000 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-data" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.399526 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-socket" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.401195 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.399968 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231c554b-45d4-41d7-9f86-5b2255306400-istiod-ca-cert" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:33:53.403093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.402929 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-certs" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.404153 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.403840 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-credential-socket" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.408759 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.407922 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-kube-api-access-k8kkd" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "kube-api-access-k8kkd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:53.408759 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.408550 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/231c554b-45d4-41d7-9f86-5b2255306400-istio-podinfo" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:33:53.408759 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.408635 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-istio-token" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:53.409619 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.409572 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-envoy" pod "231c554b-45d4-41d7-9f86-5b2255306400" (UID: "231c554b-45d4-41d7-9f86-5b2255306400"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479579 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479606 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479622 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231c554b-45d4-41d7-9f86-5b2255306400-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479639 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8kkd\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-kube-api-access-k8kkd\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479654 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479670 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479685 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231c554b-45d4-41d7-9f86-5b2255306400-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479722 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231c554b-45d4-41d7-9f86-5b2255306400-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.490982 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.479736 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231c554b-45d4-41d7-9f86-5b2255306400-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:53.613115 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.613010 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:53.620420 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.618683 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" event={"ID":"231c554b-45d4-41d7-9f86-5b2255306400","Type":"ContainerDied","Data":"f6c9dfc061a4722f5ea006d11fd5f8c9593485505f548958049928b334cd95a5"} Oct 30 19:33:53.620420 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.618776 5627 scope.go:122] "RemoveContainer" containerID="753594791f9e05a2158d446691afa3b01b049bfad17ba977e3dd084ef1a50d55" Oct 30 19:33:53.620420 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.618927 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r" Oct 30 19:33:53.628313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.626389 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" event={"ID":"1c6b620b-3265-4db3-8789-d510ddc149d7","Type":"ContainerDied","Data":"c2445c3f4a4d65b8b4a577f7c67f137e1f5647a505b4e10edce63dfad2b67609"} Oct 30 19:33:53.628313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.626541 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4" Oct 30 19:33:53.652553 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.639053 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" event={"ID":"e1893e40-3612-4b59-ad5b-ec8144073ea4","Type":"ContainerDied","Data":"a7741716ded86f25ad2d9190e33a0046e6e71bb6952696a7e8e62f10e5cf9ea0"} Oct 30 19:33:53.652553 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.639274 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m" Oct 30 19:33:53.652553 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.647577 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r"] Oct 30 19:33:53.652553 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.651978 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-sdv9r"] Oct 30 19:33:53.688354 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.687416 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4"] Oct 30 19:33:53.688354 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.687468 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-pd2n4"] Oct 30 19:33:53.689105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.688686 5627 scope.go:122] "RemoveContainer" containerID="cef44b9b2ea958e9941b52967d54e86a9e71e8a8de7ba26fc12f7dff76fda398" Oct 30 19:33:53.699978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.699953 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m"] Oct 30 19:33:53.703821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.703785 5627 scope.go:122] "RemoveContainer" containerID="eb902bf9beeabf084b5221252bcc7559ce007c95417f5dea657333387b5fd414" Oct 30 19:33:53.704833 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:53.704813 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-qqq8m"] Oct 30 19:33:55.397290 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:55.397258 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" path="/var/lib/kubelet/pods/1c6b620b-3265-4db3-8789-d510ddc149d7/volumes" Oct 30 19:33:55.397775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:55.397758 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="231c554b-45d4-41d7-9f86-5b2255306400" path="/var/lib/kubelet/pods/231c554b-45d4-41d7-9f86-5b2255306400/volumes" Oct 30 19:33:55.398187 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:55.398172 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" path="/var/lib/kubelet/pods/e1893e40-3612-4b59-ad5b-ec8144073ea4/volumes" Oct 30 19:33:57.419222 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:57.419155 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:57.546527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:57.546491 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:57.680841 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:57.680752 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:57.882060 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:33:57.882019 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:04.034516 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:04.034477 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:10.403021 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:34:10.402990 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:34:25.492279 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:25.492239 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:27.423023 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:27.422985 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:39.483406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:39.483381 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:34:39.486198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:39.486175 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:34:40.047116 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:40.047081 5627 scope.go:122] "RemoveContainer" containerID="227ceed5896dbc42feb464dd9c958a10882535acf86ce3578716bce11661112f" Oct 30 19:34:40.403780 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:34:40.403677 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:34:45.899926 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:45.899897 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:52.023194 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:52.023098 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:52.181904 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:34:52.181862 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:10.402494 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:35:10.402457 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:35:40.410632 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:35:40.410585 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:35:45.942937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:35:45.942894 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:59.224150 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:35:59.224112 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:36:10.403355 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:36:10.403315 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:36:40.402644 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:36:40.402605 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:36:46.057258 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:36:46.057221 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:36:46.340792 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:36:46.334031 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:37:10.402470 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:37:10.402439 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:37:19.469049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:37:19.469015 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:37:40.403119 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:37:40.403081 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:37:46.120715 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:37:46.120672 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:38:10.403150 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:38:10.403110 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:38:16.386419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:16.385875 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:38:16.386419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:16.385981 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:38:16.447741 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:16.446803 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:38:16.447741 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:16.446885 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:38:33.047004 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.046965 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv"] Oct 30 19:38:33.047582 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.047165 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" podUID="a7b3c348-5b38-4a00-b541-36ecdda3b032" containerName="discovery" containerID="cri-o://2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3" gracePeriod=30 Oct 30 19:38:33.292798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.292772 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:38:33.325174 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325121 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b3c348-5b38-4a00-b541-36ecdda3b032-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b3c348-5b38-4a00-b541-36ecdda3b032-local-certs\") pod \"a7b3c348-5b38-4a00-b541-36ecdda3b032\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " Oct 30 19:38:33.325174 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325150 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-token\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-token\") pod \"a7b3c348-5b38-4a00-b541-36ecdda3b032\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " Oct 30 19:38:33.325174 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325171 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-cacerts\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-cacerts\") pod \"a7b3c348-5b38-4a00-b541-36ecdda3b032\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " Oct 30 19:38:33.325412 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325197 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-ca-configmap\") pod \"a7b3c348-5b38-4a00-b541-36ecdda3b032\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " Oct 30 19:38:33.325412 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325373 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-kubeconfig\") pod \"a7b3c348-5b38-4a00-b541-36ecdda3b032\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " Oct 30 19:38:33.325572 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325542 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-dns-cert\") pod \"a7b3c348-5b38-4a00-b541-36ecdda3b032\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " Oct 30 19:38:33.325676 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325575 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-ca-configmap" pod "a7b3c348-5b38-4a00-b541-36ecdda3b032" (UID: "a7b3c348-5b38-4a00-b541-36ecdda3b032"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:38:33.325676 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325613 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-kube-api-access-kjhqq\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-kube-api-access-kjhqq\") pod \"a7b3c348-5b38-4a00-b541-36ecdda3b032\" (UID: \"a7b3c348-5b38-4a00-b541-36ecdda3b032\") " Oct 30 19:38:33.326362 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.325992 5627 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-ca-configmap\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:33.328298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.328273 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b3c348-5b38-4a00-b541-36ecdda3b032-local-certs" pod "a7b3c348-5b38-4a00-b541-36ecdda3b032" (UID: "a7b3c348-5b38-4a00-b541-36ecdda3b032"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:38:33.328407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.328367 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-kubeconfig" pod "a7b3c348-5b38-4a00-b541-36ecdda3b032" (UID: "a7b3c348-5b38-4a00-b541-36ecdda3b032"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:38:33.328407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.328389 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-dns-cert" pod "a7b3c348-5b38-4a00-b541-36ecdda3b032" (UID: "a7b3c348-5b38-4a00-b541-36ecdda3b032"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:38:33.328490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.328422 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-kube-api-access-kjhqq" pod "a7b3c348-5b38-4a00-b541-36ecdda3b032" (UID: "a7b3c348-5b38-4a00-b541-36ecdda3b032"). InnerVolumeSpecName "kube-api-access-kjhqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:38:33.328490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.328422 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-token" pod "a7b3c348-5b38-4a00-b541-36ecdda3b032" (UID: "a7b3c348-5b38-4a00-b541-36ecdda3b032"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:38:33.328727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.328685 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-cacerts" pod "a7b3c348-5b38-4a00-b541-36ecdda3b032" (UID: "a7b3c348-5b38-4a00-b541-36ecdda3b032"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:38:33.426866 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.426839 5627 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-csr-dns-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:33.426866 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.426864 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjhqq\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-kube-api-access-kjhqq\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:33.426999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.426873 5627 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b3c348-5b38-4a00-b541-36ecdda3b032-local-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:33.426999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.426887 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:33.426999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.426899 5627 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-cacerts\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:33.426999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:33.426909 5627 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a7b3c348-5b38-4a00-b541-36ecdda3b032-istio-kubeconfig\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:34.129976 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.129943 5627 generic.go:356] "Generic (PLEG): container finished" podID="a7b3c348-5b38-4a00-b541-36ecdda3b032" containerID="2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3" exitCode=0 Oct 30 19:38:34.130459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.129989 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" Oct 30 19:38:34.130459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.129999 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" event={"ID":"a7b3c348-5b38-4a00-b541-36ecdda3b032","Type":"ContainerDied","Data":"2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3"} Oct 30 19:38:34.130459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.130050 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv" event={"ID":"a7b3c348-5b38-4a00-b541-36ecdda3b032","Type":"ContainerDied","Data":"e2c63de1aab5663dbeb8089cb6c6ce7c30729e6e251fd87f043003be9712160b"} Oct 30 19:38:34.130459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.130133 5627 scope.go:122] "RemoveContainer" containerID="2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3" Oct 30 19:38:34.139334 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.139314 5627 scope.go:122] "RemoveContainer" containerID="2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3" Oct 30 19:38:34.139599 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:38:34.139580 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3\": container with ID starting with 2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3 not found: ID does not exist" containerID="2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3" Oct 30 19:38:34.139658 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.139612 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3"} err="failed to get container status \"2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3\": rpc error: code = NotFound desc = could not find container \"2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3\": container with ID starting with 2a69beba4b94c5258d351298eec7596e9b8e79181e16c5c1b3e0419fb8dc41a3 not found: ID does not exist" Oct 30 19:38:34.150914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.150889 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv"] Oct 30 19:38:34.158306 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:34.158288 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mtqgv"] Oct 30 19:38:35.397412 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:35.397386 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7b3c348-5b38-4a00-b541-36ecdda3b032" path="/var/lib/kubelet/pods/a7b3c348-5b38-4a00-b541-36ecdda3b032/volumes" Oct 30 19:38:40.403480 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:38:40.403396 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:38:46.171537 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:46.171503 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:38:58.639558 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:58.639504 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:38:59.097890 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:59.097860 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:38:59.380373 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:38:59.380266 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:02.958570 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:02.958496 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:03.096798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:03.096767 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:04.133518 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.133489 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p"] Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134027 5627 cpu_manager.go:414] "containerMap: removing container" podUID="8e3f421b-8c94-4018-92c1-f5655816d945" containerName="payload-processing" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134047 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e3f421b-8c94-4018-92c1-f5655816d945" containerName="payload-processing" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134065 5627 cpu_manager.go:414] "containerMap: removing container" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134073 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134088 5627 cpu_manager.go:414] "containerMap: removing container" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134096 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134106 5627 cpu_manager.go:414] "containerMap: removing container" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134113 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134122 5627 cpu_manager.go:414] "containerMap: removing container" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" Oct 30 19:39:04.134131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134129 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134138 5627 cpu_manager.go:414] "containerMap: removing container" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134146 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134155 5627 cpu_manager.go:414] "containerMap: removing container" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134163 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134173 5627 cpu_manager.go:414] "containerMap: removing container" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134180 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134200 5627 cpu_manager.go:414] "containerMap: removing container" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134207 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134215 5627 cpu_manager.go:414] "containerMap: removing container" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134222 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134233 5627 cpu_manager.go:414] "containerMap: removing container" podUID="d18b7b86-e8ee-4b9f-a695-92080e70f540" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134240 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d18b7b86-e8ee-4b9f-a695-92080e70f540" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134251 5627 cpu_manager.go:414] "containerMap: removing container" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134258 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134267 5627 cpu_manager.go:414] "containerMap: removing container" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134273 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134284 5627 cpu_manager.go:414] "containerMap: removing container" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134290 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134299 5627 cpu_manager.go:414] "containerMap: removing container" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134306 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134317 5627 cpu_manager.go:414] "containerMap: removing container" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134324 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134334 5627 cpu_manager.go:414] "containerMap: removing container" podUID="a7b3c348-5b38-4a00-b541-36ecdda3b032" containerName="discovery" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134341 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b3c348-5b38-4a00-b541-36ecdda3b032" containerName="discovery" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134352 5627 cpu_manager.go:414] "containerMap: removing container" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134360 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134372 5627 cpu_manager.go:414] "containerMap: removing container" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134378 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134388 5627 cpu_manager.go:414] "containerMap: removing container" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134395 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134408 5627 cpu_manager.go:414] "containerMap: removing container" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134415 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134428 5627 cpu_manager.go:414] "containerMap: removing container" podUID="71a3c20c-9338-4bce-a358-33a1d752ff46" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134435 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71a3c20c-9338-4bce-a358-33a1d752ff46" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134445 5627 cpu_manager.go:414] "containerMap: removing container" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134452 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134466 5627 cpu_manager.go:414] "containerMap: removing container" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134473 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134482 5627 cpu_manager.go:414] "containerMap: removing container" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134489 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134501 5627 cpu_manager.go:414] "containerMap: removing container" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134508 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134518 5627 cpu_manager.go:414] "containerMap: removing container" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134524 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134536 5627 cpu_manager.go:414] "containerMap: removing container" podUID="1668db65-90fe-4e2f-9ae9-70c50a2c788a" containerName="payload-processing" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134544 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1668db65-90fe-4e2f-9ae9-70c50a2c788a" containerName="payload-processing" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134554 5627 cpu_manager.go:414] "containerMap: removing container" podUID="512cbb25-1c9d-4419-a15c-d954feb58d31" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134561 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="512cbb25-1c9d-4419-a15c-d954feb58d31" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134570 5627 cpu_manager.go:414] "containerMap: removing container" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134576 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134585 5627 cpu_manager.go:414] "containerMap: removing container" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134592 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134605 5627 cpu_manager.go:414] "containerMap: removing container" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134612 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134626 5627 cpu_manager.go:414] "containerMap: removing container" podUID="e75aa317-1733-4e9f-af92-163add3ff6ab" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134632 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e75aa317-1733-4e9f-af92-163add3ff6ab" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134642 5627 cpu_manager.go:414] "containerMap: removing container" podUID="e8d59d58-fa19-4884-9d5a-5583b2758491" containerName="maas-api" Oct 30 19:39:04.134682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134648 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8d59d58-fa19-4884-9d5a-5583b2758491" containerName="maas-api" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134761 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca751225-aab9-4454-ab45-f9ace259a4ff" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134773 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="cca2f8e9-526d-4ed8-ac8e-b5bb46d18f17" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134784 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="093f3b61-0d77-473c-be63-9b96facf3b61" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134795 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134803 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="c35eab54-26ba-4fd5-958f-1e69cb47aae0" containerName="main" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134811 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="d18b7b86-e8ee-4b9f-a695-92080e70f540" containerName="maas-api" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134819 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="09f38231-eeaf-48ad-b3d4-cfe1d497418c" containerName="main" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134826 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="568a6fd2-2a37-40cf-a335-be350847e9ea" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134832 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="2350a8f2-ee45-47dc-aca4-138b0a907df8" containerName="main" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134841 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="46d47944-f78b-45f4-a00e-094ddb1ce717" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134850 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbe1332a-d689-4113-a3a5-b16e82f1f167" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134857 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b3c348-5b38-4a00-b541-36ecdda3b032" containerName="discovery" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134865 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bc6c5c7-8fcb-436d-a86b-4d9f31fd13db" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134874 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="5be4340f-16bb-46ad-8117-4d31a32aa2c1" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134880 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebf02d4a-73a3-4e29-9504-64c516a4869a" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134889 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="14e2a9bf-d40f-48b8-9fd1-0b3e67ca7317" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134897 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="71a3c20c-9338-4bce-a358-33a1d752ff46" containerName="maas-api" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134905 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="b56101b9-7ef6-4d44-a239-e509c2c49ba3" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134914 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134922 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="231c554b-45d4-41d7-9f86-5b2255306400" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134929 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="57d48892-90cc-4a09-80f8-3bd0d2c3a7b3" containerName="main" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134937 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c6b620b-3265-4db3-8789-d510ddc149d7" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134945 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2e5a633-4a42-4a8d-9eae-c7805c2cdab2" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134956 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="1668db65-90fe-4e2f-9ae9-70c50a2c788a" containerName="payload-processing" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134964 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="512cbb25-1c9d-4419-a15c-d954feb58d31" containerName="maas-api" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134971 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ea159b5-d465-4a12-83ec-ad86afa91fec" containerName="main" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134978 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2127368-191b-4224-83c9-b43f8b95e582" containerName="main" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134985 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="4989abce-250d-49ac-9fbb-b45d09a852a2" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.134994 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="e75aa317-1733-4e9f-af92-163add3ff6ab" containerName="maas-api" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.135001 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8d59d58-fa19-4884-9d5a-5583b2758491" containerName="maas-api" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.135037 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e3f421b-8c94-4018-92c1-f5655816d945" containerName="payload-processing" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.135048 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1893e40-3612-4b59-ad5b-ec8144073ea4" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.135058 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="49657d5c-b0b6-4abd-9a6f-dc8b3071e212" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.135068 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f60c8f0-fc60-4bde-b5d0-7c6ba45cefa3" containerName="istio-proxy" Oct 30 19:39:04.137678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.135077 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" Oct 30 19:39:04.138948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.138310 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.147834 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.147810 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-88ec2c-serving-cert\"" Oct 30 19:39:04.154586 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.154563 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p"] Oct 30 19:39:04.195983 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.195945 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:04.208204 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.208175 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-metrics-certs\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.208330 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.208230 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkx6s\" (UniqueName: \"kubernetes.io/projected/abbdba4f-918c-4a11-9562-46522d882a8a-kube-api-access-xkx6s\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.208374 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.208330 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.208411 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.208393 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbdba4f-918c-4a11-9562-46522d882a8a-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.267495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.267458 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89"] Oct 30 19:39:04.267925 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.267905 5627 cpu_manager.go:414] "containerMap: removing container" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" Oct 30 19:39:04.268036 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.267927 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77a4921d-b562-4871-b94c-ae5896d7a997" containerName="istio-proxy" Oct 30 19:39:04.271078 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.271059 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:04.277490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.277464 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-c55a28be-dockercfg-9xmhr\"" Oct 30 19:39:04.288824 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.288801 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj"] Oct 30 19:39:04.292283 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.292266 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89"] Oct 30 19:39:04.292368 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.292340 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:04.303852 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.303823 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj"] Oct 30 19:39:04.309496 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.309466 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbdba4f-918c-4a11-9562-46522d882a8a-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.309603 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.309525 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-metrics-certs\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.309603 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.309577 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkx6s\" (UniqueName: \"kubernetes.io/projected/abbdba4f-918c-4a11-9562-46522d882a8a-kube-api-access-xkx6s\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.309737 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.309626 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.310238 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.310214 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbdba4f-918c-4a11-9562-46522d882a8a-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.312326 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.312304 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.312884 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.312862 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-metrics-certs\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.321165 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.321145 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkx6s\" (UniqueName: \"kubernetes.io/projected/abbdba4f-918c-4a11-9562-46522d882a8a-kube-api-access-xkx6s\") pod \"maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.341115 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.341092 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-88ec2c\"" Oct 30 19:39:04.360251 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.360215 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:04.410393 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.410317 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89\" (UID: \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:04.410551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.410459 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr7vj\" (UniqueName: \"kubernetes.io/projected/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-kube-api-access-xr7vj\") pod \"payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89\" (UID: \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:04.410551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.410500 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh5h4\" (UniqueName: \"kubernetes.io/projected/a88a018f-6f5e-4444-9208-4a245eb642bc-kube-api-access-lh5h4\") pod \"payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj\" (UID: \"a88a018f-6f5e-4444-9208-4a245eb642bc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:04.410669 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.410616 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88a018f-6f5e-4444-9208-4a245eb642bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj\" (UID: \"a88a018f-6f5e-4444-9208-4a245eb642bc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:04.453214 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.453190 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:04.511403 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.511375 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr7vj\" (UniqueName: \"kubernetes.io/projected/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-kube-api-access-xr7vj\") pod \"payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89\" (UID: \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:04.511549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.511423 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh5h4\" (UniqueName: \"kubernetes.io/projected/a88a018f-6f5e-4444-9208-4a245eb642bc-kube-api-access-lh5h4\") pod \"payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj\" (UID: \"a88a018f-6f5e-4444-9208-4a245eb642bc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:04.511549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.511493 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88a018f-6f5e-4444-9208-4a245eb642bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj\" (UID: \"a88a018f-6f5e-4444-9208-4a245eb642bc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:04.511549 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.511537 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89\" (UID: \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:04.512298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.512276 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88a018f-6f5e-4444-9208-4a245eb642bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj\" (UID: \"a88a018f-6f5e-4444-9208-4a245eb642bc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:04.512370 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.512313 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89\" (UID: \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:04.524526 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.524492 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr7vj\" (UniqueName: \"kubernetes.io/projected/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-kube-api-access-xr7vj\") pod \"payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89\" (UID: \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:04.524642 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.524561 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh5h4\" (UniqueName: \"kubernetes.io/projected/a88a018f-6f5e-4444-9208-4a245eb642bc-kube-api-access-lh5h4\") pod \"payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj\" (UID: \"a88a018f-6f5e-4444-9208-4a245eb642bc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:04.583635 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.583603 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:04.605189 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.605158 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:04.761647 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.761621 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p"] Oct 30 19:39:04.763669 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:39:04.763637 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbdba4f_918c_4a11_9562_46522d882a8a.slice/crio-5c5d79a932ecb5e2c5f988af313efa43c72124fb551d24ea71cdd90710820fbf WatchSource:0}: Error finding container 5c5d79a932ecb5e2c5f988af313efa43c72124fb551d24ea71cdd90710820fbf: Status 404 returned error can't find the container with id 5c5d79a932ecb5e2c5f988af313efa43c72124fb551d24ea71cdd90710820fbf Oct 30 19:39:04.765174 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.765152 5627 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:39:04.877322 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.877295 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89"] Oct 30 19:39:04.880579 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:39:04.880548 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4018a2_5cb8_43d0_9d13_bd52a0d2ebff.slice/crio-1f95981b8a79da892a0971d17be8dcfb40ac63fd3b36ec2ac3b1a0e44e9547f5 WatchSource:0}: Error finding container 1f95981b8a79da892a0971d17be8dcfb40ac63fd3b36ec2ac3b1a0e44e9547f5: Status 404 returned error can't find the container with id 1f95981b8a79da892a0971d17be8dcfb40ac63fd3b36ec2ac3b1a0e44e9547f5 Oct 30 19:39:04.904937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:04.904916 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj"] Oct 30 19:39:04.911794 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:39:04.911764 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88a018f_6f5e_4444_9208_4a245eb642bc.slice/crio-209617490c0abcc0c68217eed82e6aa84810ba5c7418373a1bb1202bc128673e WatchSource:0}: Error finding container 209617490c0abcc0c68217eed82e6aa84810ba5c7418373a1bb1202bc128673e: Status 404 returned error can't find the container with id 209617490c0abcc0c68217eed82e6aa84810ba5c7418373a1bb1202bc128673e Oct 30 19:39:05.271831 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.271790 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" event={"ID":"abbdba4f-918c-4a11-9562-46522d882a8a","Type":"ContainerStarted","Data":"98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202"} Oct 30 19:39:05.272317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.271881 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" event={"ID":"abbdba4f-918c-4a11-9562-46522d882a8a","Type":"ContainerStarted","Data":"5c5d79a932ecb5e2c5f988af313efa43c72124fb551d24ea71cdd90710820fbf"} Oct 30 19:39:05.272317 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.271909 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:05.274099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.274067 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" event={"ID":"a88a018f-6f5e-4444-9208-4a245eb642bc","Type":"ContainerStarted","Data":"12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2"} Oct 30 19:39:05.274219 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.274122 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:05.274219 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.274139 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" event={"ID":"a88a018f-6f5e-4444-9208-4a245eb642bc","Type":"ContainerStarted","Data":"209617490c0abcc0c68217eed82e6aa84810ba5c7418373a1bb1202bc128673e"} Oct 30 19:39:05.275709 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.275679 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" event={"ID":"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff","Type":"ContainerStarted","Data":"5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c"} Oct 30 19:39:05.275794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.275756 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" event={"ID":"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff","Type":"ContainerStarted","Data":"1f95981b8a79da892a0971d17be8dcfb40ac63fd3b36ec2ac3b1a0e44e9547f5"} Oct 30 19:39:05.275871 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.275848 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:05.287150 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.287128 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:05.294209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.294173 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" podStartSLOduration=0.899674531 podStartE2EDuration="1.29416154s" totalImagesPullingTime="394.487009ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:05.292764365 +0000 UTC m=+1166.019581661" watchObservedRunningTime="2026-10-30 19:39:05.29416154 +0000 UTC m=+1166.020978836" Oct 30 19:39:05.316871 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.316819 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" podStartSLOduration=1.316805631 podStartE2EDuration="1.316805631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:05.314996444 +0000 UTC m=+1166.041813741" watchObservedRunningTime="2026-10-30 19:39:05.316805631 +0000 UTC m=+1166.043622928" Oct 30 19:39:05.336401 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:05.336356 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" podStartSLOduration=1.336344767 podStartE2EDuration="1.336344767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:05.336076608 +0000 UTC m=+1166.062893906" watchObservedRunningTime="2026-10-30 19:39:05.336344767 +0000 UTC m=+1166.063162063" Oct 30 19:39:06.104978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:06.104941 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:06.797344 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:06.797297 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:07.656014 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:07.655976 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:08.398384 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:08.398346 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:09.202457 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:09.202386 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:10.089686 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:10.089633 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:10.404139 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:39:10.404059 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:39:10.901837 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:10.901801 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:11.284925 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:11.284895 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:39:11.890798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:11.890765 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:12.832148 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:12.832117 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:12.961895 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:12.961864 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:13.489523 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:13.489492 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:13.650445 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:13.650414 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:13.842954 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:13.842840 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:14.849926 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:14.849884 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:15.516179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:15.516129 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:15.834360 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:15.834273 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:16.283697 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:16.283662 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:39:16.284183 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:16.283762 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:39:16.695650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:16.695576 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:17.698233 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:17.698200 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:18.635242 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:18.635205 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:19.519794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:19.519768 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:20.518941 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:20.518875 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:21.292980 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:21.292938 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:22.116715 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:22.116681 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:22.901104 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:22.901065 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:23.718408 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:23.718368 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:24.613305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:24.613260 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:25.305173 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:25.305139 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:26.205986 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:26.205946 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:27.022453 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:27.022419 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:27.809519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:27.809478 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:28.616846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:28.616809 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:29.376084 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:29.376020 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:30.159810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:30.159772 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:30.970652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:30.970156 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:31.701656 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:31.701619 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:32.585350 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:32.585312 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:33.396817 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:33.396785 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:34.180185 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:34.180150 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:35.028277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:35.028240 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:35.716995 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:35.716962 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:36.496662 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:36.496616 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:37.403591 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:37.403540 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:38.301725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:38.301670 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:38.989737 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:38.989668 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:39.510404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:39.510373 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:39:39.513655 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:39.513634 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:39:39.725040 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:39.725009 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:40.403810 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:39:40.403772 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:39:40.492879 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:40.492835 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:41.169345 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:41.169311 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:41.924299 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:41.924263 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:42.695182 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:42.695147 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:43.508826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:43.508791 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:44.294582 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:44.294549 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:45.097847 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:45.097815 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:45.931664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:45.931635 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:46.231057 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:46.230976 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:46.711373 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:46.711346 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:47.612939 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:47.612903 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:48.308369 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:48.308317 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:49.004718 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:49.004666 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:49.772780 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:49.772732 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:50.603119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:50.603083 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:51.315069 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:51.313937 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:52.058874 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:52.058845 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:52.789318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:52.789281 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:53.494571 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:53.494511 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:54.195112 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:54.195072 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:54.973679 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:54.973647 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:55.674555 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:55.674518 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:55.874815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:55.874778 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:56.797983 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:56.797953 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:57.716178 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:57.716148 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:58.486286 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:58.486255 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:59.270265 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:39:59.270229 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:00.097100 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:00.097068 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:00.650456 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:00.650423 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:00.796565 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:00.796535 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:01.496286 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:01.496249 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:02.388759 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:02.388730 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:03.102527 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:03.102482 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:03.794959 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:03.794929 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:04.526432 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:04.526395 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:05.272332 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:05.272294 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:05.812362 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:05.812321 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p"] Oct 30 19:40:05.812785 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:05.812596 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" podUID="abbdba4f-918c-4a11-9562-46522d882a8a" containerName="maas-api" containerID="cri-o://98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202" gracePeriod=30 Oct 30 19:40:05.903126 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:05.903093 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89"] Oct 30 19:40:05.903888 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:05.903864 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:05.908520 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:05.907903 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj"] Oct 30 19:40:06.032261 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.032228 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:06.083248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.083210 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:40:06.102879 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.102845 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abbdba4f-918c-4a11-9562-46522d882a8a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbdba4f-918c-4a11-9562-46522d882a8a-openshift-service-ca\") pod \"abbdba4f-918c-4a11-9562-46522d882a8a\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " Oct 30 19:40:06.103132 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.102943 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-maas-api-tls\") pod \"abbdba4f-918c-4a11-9562-46522d882a8a\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " Oct 30 19:40:06.103132 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.102983 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-metrics-certs\") pod \"abbdba4f-918c-4a11-9562-46522d882a8a\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " Oct 30 19:40:06.103329 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.103129 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abbdba4f-918c-4a11-9562-46522d882a8a-kube-api-access-xkx6s\" (UniqueName: \"kubernetes.io/projected/abbdba4f-918c-4a11-9562-46522d882a8a-kube-api-access-xkx6s\") pod \"abbdba4f-918c-4a11-9562-46522d882a8a\" (UID: \"abbdba4f-918c-4a11-9562-46522d882a8a\") " Oct 30 19:40:06.104454 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.104419 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abbdba4f-918c-4a11-9562-46522d882a8a-openshift-service-ca" pod "abbdba4f-918c-4a11-9562-46522d882a8a" (UID: "abbdba4f-918c-4a11-9562-46522d882a8a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:40:06.106763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.106689 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abbdba4f-918c-4a11-9562-46522d882a8a-kube-api-access-xkx6s" pod "abbdba4f-918c-4a11-9562-46522d882a8a" (UID: "abbdba4f-918c-4a11-9562-46522d882a8a"). InnerVolumeSpecName "kube-api-access-xkx6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:40:06.107277 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.107223 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-metrics-certs" pod "abbdba4f-918c-4a11-9562-46522d882a8a" (UID: "abbdba4f-918c-4a11-9562-46522d882a8a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:40:06.110137 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.110100 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-maas-api-tls" pod "abbdba4f-918c-4a11-9562-46522d882a8a" (UID: "abbdba4f-918c-4a11-9562-46522d882a8a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:40:06.204857 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.204824 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkx6s\" (UniqueName: \"kubernetes.io/projected/abbdba4f-918c-4a11-9562-46522d882a8a-kube-api-access-xkx6s\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:40:06.204857 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.204851 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbdba4f-918c-4a11-9562-46522d882a8a-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:40:06.205089 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.204862 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:40:06.205089 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.204874 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbdba4f-918c-4a11-9562-46522d882a8a-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:40:06.342177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.342068 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:06.471577 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.471545 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:06.558503 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.558464 5627 generic.go:356] "Generic (PLEG): container finished" podID="abbdba4f-918c-4a11-9562-46522d882a8a" containerID="98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202" exitCode=0 Oct 30 19:40:06.558677 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.558514 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" Oct 30 19:40:06.558677 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.558554 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" event={"ID":"abbdba4f-918c-4a11-9562-46522d882a8a","Type":"ContainerDied","Data":"98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202"} Oct 30 19:40:06.558677 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.558607 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p" event={"ID":"abbdba4f-918c-4a11-9562-46522d882a8a","Type":"ContainerDied","Data":"5c5d79a932ecb5e2c5f988af313efa43c72124fb551d24ea71cdd90710820fbf"} Oct 30 19:40:06.558677 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.558628 5627 scope.go:122] "RemoveContainer" containerID="98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202" Oct 30 19:40:06.568579 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.568562 5627 scope.go:122] "RemoveContainer" containerID="98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202" Oct 30 19:40:06.568877 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:40:06.568860 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202\": container with ID starting with 98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202 not found: ID does not exist" containerID="98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202" Oct 30 19:40:06.568943 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.568884 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202"} err="failed to get container status \"98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202\": rpc error: code = NotFound desc = could not find container \"98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202\": container with ID starting with 98dd103a82f548a6e16eb416ea254b2b62278118668488d9520e326f353b9202 not found: ID does not exist" Oct 30 19:40:06.583775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.583732 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p"] Oct 30 19:40:06.589061 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.589039 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-88ec2c-579748589c-rqt7p"] Oct 30 19:40:06.878839 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:06.878810 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:07.397210 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:07.397176 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abbdba4f-918c-4a11-9562-46522d882a8a" path="/var/lib/kubelet/pods/abbdba4f-918c-4a11-9562-46522d882a8a/volumes" Oct 30 19:40:10.404454 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:40:10.404415 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:40:10.903634 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:10.903573 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" podUID="da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" containerName="payload-processing" containerID="cri-o://5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c" gracePeriod=25 Oct 30 19:40:10.909115 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:10.909085 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" podUID="a88a018f-6f5e-4444-9208-4a245eb642bc" containerName="payload-pre-processing" containerID="cri-o://12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2" gracePeriod=25 Oct 30 19:40:13.203652 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:13.203615 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:13.345334 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:13.345304 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:13.582964 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:13.582929 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:16.282158 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:16.282119 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" podUID="a88a018f-6f5e-4444-9208-4a245eb642bc" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:40:16.282546 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:16.282160 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" podUID="da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:40:26.282358 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:26.282321 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" podUID="da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:40:26.282358 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:26.282346 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" podUID="a88a018f-6f5e-4444-9208-4a245eb642bc" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:40:31.152559 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.152537 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:40:31.182202 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.182180 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:40:31.241178 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.241151 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a88a018f-6f5e-4444-9208-4a245eb642bc-kube-api-access-lh5h4\" (UniqueName: \"kubernetes.io/projected/a88a018f-6f5e-4444-9208-4a245eb642bc-kube-api-access-lh5h4\") pod \"a88a018f-6f5e-4444-9208-4a245eb642bc\" (UID: \"a88a018f-6f5e-4444-9208-4a245eb642bc\") " Oct 30 19:40:31.241298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.241197 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-plugins-config-volume\") pod \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\" (UID: \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\") " Oct 30 19:40:31.241344 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.241312 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-kube-api-access-xr7vj\" (UniqueName: \"kubernetes.io/projected/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-kube-api-access-xr7vj\") pod \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\" (UID: \"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff\") " Oct 30 19:40:31.241391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.241375 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a88a018f-6f5e-4444-9208-4a245eb642bc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88a018f-6f5e-4444-9208-4a245eb642bc-plugins-config-volume\") pod \"a88a018f-6f5e-4444-9208-4a245eb642bc\" (UID: \"a88a018f-6f5e-4444-9208-4a245eb642bc\") " Oct 30 19:40:31.241770 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.241746 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-plugins-config-volume" pod "da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" (UID: "da4018a2-5cb8-43d0-9d13-bd52a0d2ebff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:40:31.241770 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.241748 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a88a018f-6f5e-4444-9208-4a245eb642bc-plugins-config-volume" pod "a88a018f-6f5e-4444-9208-4a245eb642bc" (UID: "a88a018f-6f5e-4444-9208-4a245eb642bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:40:31.241897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.241825 5627 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-plugins-config-volume\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:40:31.241897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.241843 5627 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88a018f-6f5e-4444-9208-4a245eb642bc-plugins-config-volume\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:40:31.243455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.243435 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a88a018f-6f5e-4444-9208-4a245eb642bc-kube-api-access-lh5h4" pod "a88a018f-6f5e-4444-9208-4a245eb642bc" (UID: "a88a018f-6f5e-4444-9208-4a245eb642bc"). InnerVolumeSpecName "kube-api-access-lh5h4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:40:31.243518 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.243477 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-kube-api-access-xr7vj" pod "da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" (UID: "da4018a2-5cb8-43d0-9d13-bd52a0d2ebff"). InnerVolumeSpecName "kube-api-access-xr7vj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:40:31.342643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.342620 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr7vj\" (UniqueName: \"kubernetes.io/projected/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff-kube-api-access-xr7vj\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:40:31.342643 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.342641 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh5h4\" (UniqueName: \"kubernetes.io/projected/a88a018f-6f5e-4444-9208-4a245eb642bc-kube-api-access-lh5h4\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:40:31.687821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.687786 5627 generic.go:356] "Generic (PLEG): container finished" podID="da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" containerID="5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c" exitCode=0 Oct 30 19:40:31.687983 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.687829 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" Oct 30 19:40:31.687983 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.687871 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" event={"ID":"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff","Type":"ContainerDied","Data":"5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c"} Oct 30 19:40:31.687983 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.687918 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89" event={"ID":"da4018a2-5cb8-43d0-9d13-bd52a0d2ebff","Type":"ContainerDied","Data":"1f95981b8a79da892a0971d17be8dcfb40ac63fd3b36ec2ac3b1a0e44e9547f5"} Oct 30 19:40:31.687983 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.687939 5627 scope.go:122] "RemoveContainer" containerID="5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c" Oct 30 19:40:31.689294 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.689269 5627 generic.go:356] "Generic (PLEG): container finished" podID="a88a018f-6f5e-4444-9208-4a245eb642bc" containerID="12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2" exitCode=0 Oct 30 19:40:31.689419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.689314 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" Oct 30 19:40:31.689419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.689349 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" event={"ID":"a88a018f-6f5e-4444-9208-4a245eb642bc","Type":"ContainerDied","Data":"12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2"} Oct 30 19:40:31.689419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.689386 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj" event={"ID":"a88a018f-6f5e-4444-9208-4a245eb642bc","Type":"ContainerDied","Data":"209617490c0abcc0c68217eed82e6aa84810ba5c7418373a1bb1202bc128673e"} Oct 30 19:40:31.699893 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.699875 5627 scope.go:122] "RemoveContainer" containerID="5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c" Oct 30 19:40:31.700199 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:40:31.700178 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c\": container with ID starting with 5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c not found: ID does not exist" containerID="5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c" Oct 30 19:40:31.700298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.700203 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c"} err="failed to get container status \"5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c\": rpc error: code = NotFound desc = could not find container \"5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c\": container with ID starting with 5f607da6e25a263002d62474bb67067f83bf5774f212a8877e4e6bfad343106c not found: ID does not exist" Oct 30 19:40:31.700298 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.700220 5627 scope.go:122] "RemoveContainer" containerID="12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2" Oct 30 19:40:31.707721 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.707690 5627 scope.go:122] "RemoveContainer" containerID="12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2" Oct 30 19:40:31.708031 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:40:31.708002 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2\": container with ID starting with 12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2 not found: ID does not exist" containerID="12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2" Oct 30 19:40:31.708031 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.708024 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2"} err="failed to get container status \"12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2\": rpc error: code = NotFound desc = could not find container \"12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2\": container with ID starting with 12badaec77f10ccb7cd05e86cc0e5351555e273820127f32da9b6a777f6a75a2 not found: ID does not exist" Oct 30 19:40:31.710198 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.710179 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89"] Oct 30 19:40:31.715497 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.715476 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-88ec2c-56d4cf478pvg89"] Oct 30 19:40:31.727965 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.727945 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj"] Oct 30 19:40:31.733449 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:31.733425 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-88ec2c-74d7chb7mj"] Oct 30 19:40:33.397172 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:33.397136 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a88a018f-6f5e-4444-9208-4a245eb642bc" path="/var/lib/kubelet/pods/a88a018f-6f5e-4444-9208-4a245eb642bc/volumes" Oct 30 19:40:33.397543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:33.397467 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" path="/var/lib/kubelet/pods/da4018a2-5cb8-43d0-9d13-bd52a0d2ebff/volumes" Oct 30 19:40:37.018626 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:37.018588 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:38.533045 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:38.533006 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:40.403492 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:40:40.403413 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:40:44.745502 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:44.745471 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:46.259500 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:40:46.259447 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:41:10.403243 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:41:10.403206 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:41:40.404067 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:41:40.404033 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:41:44.562290 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:44.562259 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:41:46.298675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:46.298644 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:41:52.833424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833382 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r"] Oct 30 19:41:52.833862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833762 5627 cpu_manager.go:414] "containerMap: removing container" podUID="abbdba4f-918c-4a11-9562-46522d882a8a" containerName="maas-api" Oct 30 19:41:52.833862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833775 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abbdba4f-918c-4a11-9562-46522d882a8a" containerName="maas-api" Oct 30 19:41:52.833862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833790 5627 cpu_manager.go:414] "containerMap: removing container" podUID="a88a018f-6f5e-4444-9208-4a245eb642bc" containerName="payload-pre-processing" Oct 30 19:41:52.833862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833796 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a88a018f-6f5e-4444-9208-4a245eb642bc" containerName="payload-pre-processing" Oct 30 19:41:52.833862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833815 5627 cpu_manager.go:414] "containerMap: removing container" podUID="da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" containerName="payload-processing" Oct 30 19:41:52.833862 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833820 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" containerName="payload-processing" Oct 30 19:41:52.834083 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833874 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="da4018a2-5cb8-43d0-9d13-bd52a0d2ebff" containerName="payload-processing" Oct 30 19:41:52.834083 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833886 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="abbdba4f-918c-4a11-9562-46522d882a8a" containerName="maas-api" Oct 30 19:41:52.834083 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.833896 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="a88a018f-6f5e-4444-9208-4a245eb642bc" containerName="payload-pre-processing" Oct 30 19:41:52.837079 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.837055 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:52.844791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.844546 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Oct 30 19:41:52.845190 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.845147 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Oct 30 19:41:52.845300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.845199 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5ptsd\"" Oct 30 19:41:52.846306 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.846279 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r"] Oct 30 19:41:52.883015 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.882988 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:41:52.938872 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.938846 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d631061b-03d9-4e53-a42d-f1ce92fc2af4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:52.938988 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.938888 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d631061b-03d9-4e53-a42d-f1ce92fc2af4-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:52.939037 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:52.939000 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vkr5\" (UniqueName: \"kubernetes.io/projected/d631061b-03d9-4e53-a42d-f1ce92fc2af4-kube-api-access-9vkr5\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:53.039665 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.039625 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vkr5\" (UniqueName: \"kubernetes.io/projected/d631061b-03d9-4e53-a42d-f1ce92fc2af4-kube-api-access-9vkr5\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:53.039859 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.039723 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d631061b-03d9-4e53-a42d-f1ce92fc2af4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:53.039859 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.039771 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d631061b-03d9-4e53-a42d-f1ce92fc2af4-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:53.040449 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.040427 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d631061b-03d9-4e53-a42d-f1ce92fc2af4-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:53.042284 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.042258 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d631061b-03d9-4e53-a42d-f1ce92fc2af4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:53.054210 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.054186 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vkr5\" (UniqueName: \"kubernetes.io/projected/d631061b-03d9-4e53-a42d-f1ce92fc2af4-kube-api-access-9vkr5\") pod \"kuadrant-console-plugin-66c4fb575d-vs77r\" (UID: \"d631061b-03d9-4e53-a42d-f1ce92fc2af4\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:53.091623 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.091554 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:41:53.151353 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.151331 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" Oct 30 19:41:53.430396 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:53.430272 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r"] Oct 30 19:41:53.433340 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:41:53.433314 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd631061b_03d9_4e53_a42d_f1ce92fc2af4.slice/crio-1d961fa70c2677ba425e7d8e65b49f978b0156140a1a6de4cc213a3fc1a3ae0b WatchSource:0}: Error finding container 1d961fa70c2677ba425e7d8e65b49f978b0156140a1a6de4cc213a3fc1a3ae0b: Status 404 returned error can't find the container with id 1d961fa70c2677ba425e7d8e65b49f978b0156140a1a6de4cc213a3fc1a3ae0b Oct 30 19:41:54.078124 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:54.078082 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" event={"ID":"d631061b-03d9-4e53-a42d-f1ce92fc2af4","Type":"ContainerStarted","Data":"1d961fa70c2677ba425e7d8e65b49f978b0156140a1a6de4cc213a3fc1a3ae0b"} Oct 30 19:41:57.179268 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:57.179228 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:41:57.331917 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:41:57.331878 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:00.128663 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:00.128630 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" event={"ID":"d631061b-03d9-4e53-a42d-f1ce92fc2af4","Type":"ContainerStarted","Data":"195eb341a11875c7481c40ee2901b5bf48744cb9ac54e3a1e3c40a9f6766bd89"} Oct 30 19:42:00.148313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:00.148271 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vs77r" podStartSLOduration=1.954002543 podStartE2EDuration="8.148259256s" totalImagesPullingTime="6.194256713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:00.147925126 +0000 UTC m=+1340.874742422" watchObservedRunningTime="2026-10-30 19:42:00.148259256 +0000 UTC m=+1340.875076552" Oct 30 19:42:10.412548 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:42:10.412506 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:42:18.821194 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:18.821156 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:27.003512 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:27.003436 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:27.276861 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:27.276783 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:27.423161 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:27.423128 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:30.471458 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:30.471398 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:30.614001 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:30.613964 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:31.641523 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.641480 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74"] Oct 30 19:42:31.645119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.645098 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.661595 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.661570 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74"] Oct 30 19:42:31.668776 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.668753 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-ad6b27-serving-cert\"" Oct 30 19:42:31.689323 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.689296 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.689447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.689333 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/613f4cd0-8edb-4fbc-9629-d40b926c2663-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.689447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.689358 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-metrics-certs\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.689547 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.689510 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmncf\" (UniqueName: \"kubernetes.io/projected/613f4cd0-8edb-4fbc-9629-d40b926c2663-kube-api-access-nmncf\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.722905 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.722878 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:31.787639 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.787611 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf"] Oct 30 19:42:31.790664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.790643 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.790781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.790673 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/613f4cd0-8edb-4fbc-9629-d40b926c2663-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.790781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.790694 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-metrics-certs\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.790781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.790738 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmncf\" (UniqueName: \"kubernetes.io/projected/613f4cd0-8edb-4fbc-9629-d40b926c2663-kube-api-access-nmncf\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.791083 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.791003 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:31.791487 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.791464 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/613f4cd0-8edb-4fbc-9629-d40b926c2663-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.793532 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.793509 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.793691 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.793673 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-metrics-certs\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.806331 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.806307 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmncf\" (UniqueName: \"kubernetes.io/projected/613f4cd0-8edb-4fbc-9629-d40b926c2663-kube-api-access-nmncf\") pod \"maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.813152 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.813130 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf"] Oct 30 19:42:31.869037 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.869009 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-0c8f07cb-dockercfg-tqhvc\"" Oct 30 19:42:31.874079 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.874052 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:31.881561 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.881543 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-ad6b27\"" Oct 30 19:42:31.891719 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.891644 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b27c6d4-f707-4af1-a435-d42b5108cb9d-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf\" (UID: \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:31.891835 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.891816 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrssw\" (UniqueName: \"kubernetes.io/projected/3b27c6d4-f707-4af1-a435-d42b5108cb9d-kube-api-access-zrssw\") pod \"payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf\" (UID: \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:31.958810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.958774 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:31.993407 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.993372 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrssw\" (UniqueName: \"kubernetes.io/projected/3b27c6d4-f707-4af1-a435-d42b5108cb9d-kube-api-access-zrssw\") pod \"payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf\" (UID: \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:31.993567 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.993471 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b27c6d4-f707-4af1-a435-d42b5108cb9d-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf\" (UID: \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:31.994510 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.994481 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b27c6d4-f707-4af1-a435-d42b5108cb9d-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf\" (UID: \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:31.994750 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:31.994696 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:32.007086 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:32.007065 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrssw\" (UniqueName: \"kubernetes.io/projected/3b27c6d4-f707-4af1-a435-d42b5108cb9d-kube-api-access-zrssw\") pod \"payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf\" (UID: \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:32.124107 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:32.124080 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:32.235807 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:32.235756 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74"] Oct 30 19:42:32.238745 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:42:32.238697 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613f4cd0_8edb_4fbc_9629_d40b926c2663.slice/crio-4b76faf113d526e37143d24419a08f8de88df8f8fdc9ded7a707b833d264b5d0 WatchSource:0}: Error finding container 4b76faf113d526e37143d24419a08f8de88df8f8fdc9ded7a707b833d264b5d0: Status 404 returned error can't find the container with id 4b76faf113d526e37143d24419a08f8de88df8f8fdc9ded7a707b833d264b5d0 Oct 30 19:42:32.279733 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:32.279675 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" event={"ID":"613f4cd0-8edb-4fbc-9629-d40b926c2663","Type":"ContainerStarted","Data":"4b76faf113d526e37143d24419a08f8de88df8f8fdc9ded7a707b833d264b5d0"} Oct 30 19:42:32.399760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:32.399733 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf"] Oct 30 19:42:32.401310 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:42:32.401282 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b27c6d4_f707_4af1_a435_d42b5108cb9d.slice/crio-06fa49333fab85119e157025831d72c358a600f67c67062110f1bc649631b44a WatchSource:0}: Error finding container 06fa49333fab85119e157025831d72c358a600f67c67062110f1bc649631b44a: Status 404 returned error can't find the container with id 06fa49333fab85119e157025831d72c358a600f67c67062110f1bc649631b44a Oct 30 19:42:32.708435 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:32.708411 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:33.286118 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:33.286078 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" event={"ID":"613f4cd0-8edb-4fbc-9629-d40b926c2663","Type":"ContainerStarted","Data":"258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb"} Oct 30 19:42:33.286320 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:33.286237 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:33.287600 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:33.287578 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" event={"ID":"3b27c6d4-f707-4af1-a435-d42b5108cb9d","Type":"ContainerStarted","Data":"d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0"} Oct 30 19:42:33.287730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:33.287620 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:33.287730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:33.287636 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" event={"ID":"3b27c6d4-f707-4af1-a435-d42b5108cb9d","Type":"ContainerStarted","Data":"06fa49333fab85119e157025831d72c358a600f67c67062110f1bc649631b44a"} Oct 30 19:42:33.310750 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:33.310692 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" podStartSLOduration=1.845455792 podStartE2EDuration="2.310679593s" totalImagesPullingTime="465.223801ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:33.30847689 +0000 UTC m=+1374.035294214" watchObservedRunningTime="2026-10-30 19:42:33.310679593 +0000 UTC m=+1374.037496890" Oct 30 19:42:33.332276 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:33.332236 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" podStartSLOduration=2.332224654 podStartE2EDuration="2.332224654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:33.32989804 +0000 UTC m=+1374.056715334" watchObservedRunningTime="2026-10-30 19:42:33.332224654 +0000 UTC m=+1374.059041948" Oct 30 19:42:33.531189 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:33.531139 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:34.336351 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:34.336306 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:35.254728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:35.254655 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:38.379444 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:38.379396 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:38.539743 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:38.539688 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:38.739473 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:38.739443 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:39.298048 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:39.298022 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:42:39.881047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:39.880979 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:40.402557 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:42:40.402518 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:42:40.628957 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:40.628915 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:41.481115 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:41.481070 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:42.372523 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:42.372488 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:43.100760 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:43.100728 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:43.899160 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:43.899121 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:44.295728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:44.295669 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:42:44.630990 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:44.630871 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:45.306196 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:45.306163 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:45.430884 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:45.430851 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:46.032943 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:46.032900 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:46.176991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:46.176960 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:46.323350 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:46.323261 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:46.461375 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:46.461334 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:47.243419 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:47.243390 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:48.423614 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:48.423560 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:49.043816 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:49.043766 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:49.371883 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:49.371789 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:50.523843 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:50.523811 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:51.689421 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:51.689384 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:52.953363 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:52.953329 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:53.806398 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:53.806363 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:54.766342 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:54.766303 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:55.581879 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:55.581841 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:56.507912 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:56.507881 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:57.310815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:57.310780 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:58.022436 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:58.022401 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:58.944285 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:58.944249 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:42:59.700152 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:42:59.700120 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:00.578338 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:00.578299 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:01.448231 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:01.448198 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:02.178234 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:02.178202 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:02.960234 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:02.960196 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:03.929208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:03.929173 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:04.819190 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:04.819153 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:05.856868 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:05.856835 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:07.014631 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:07.014595 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:07.982917 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:07.982883 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:08.786944 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:08.786911 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:09.621753 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:09.621698 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:10.403275 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:43:10.403232 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:43:10.431825 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:10.431794 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:11.201878 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:11.201840 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:11.918248 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:11.918213 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:12.673355 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:12.673327 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:13.564488 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:13.564453 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:14.326537 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:14.326462 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:15.083677 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:15.083646 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:15.816313 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:15.816281 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:16.662572 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:16.662537 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:17.591116 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:17.591082 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:18.399864 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:18.399820 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:19.201920 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:19.201890 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:19.998891 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:19.998853 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:20.701785 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:20.701752 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:21.405056 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:21.405024 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:22.199273 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:22.199237 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:22.983090 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:22.983052 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:23.702416 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:23.702381 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:24.403215 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:24.403179 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:25.155512 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:25.155480 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:25.997491 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:25.997449 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:26.791143 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:26.791109 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:27.867214 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:27.867181 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:28.590548 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:28.590511 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:29.051562 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:29.051529 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:29.793019 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:29.792981 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:30.504361 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:30.504333 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:31.372099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:31.372070 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:32.093485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:32.093448 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:32.929781 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:32.929748 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:33.650839 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:33.650801 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:33.796551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:33.796514 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:34.697752 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:34.697723 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:35.504593 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:35.504551 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:36.358441 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:36.358396 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:37.374687 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:37.374653 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:38.285365 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:38.285331 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:39.024650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:39.024613 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf"] Oct 30 19:43:39.238650 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:39.238613 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:39.388660 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:39.388583 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:39.915673 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:39.915640 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:40.404019 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:43:40.403983 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:43:40.529179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.529147 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74"] Oct 30 19:43:40.529406 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.529381 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" podUID="613f4cd0-8edb-4fbc-9629-d40b926c2663" containerName="maas-api" containerID="cri-o://258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb" gracePeriod=30 Oct 30 19:43:40.604684 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.604655 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:40.760819 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.760787 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:40.787388 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.787366 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:43:40.856500 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.856471 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/613f4cd0-8edb-4fbc-9629-d40b926c2663-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/613f4cd0-8edb-4fbc-9629-d40b926c2663-openshift-service-ca\") pod \"613f4cd0-8edb-4fbc-9629-d40b926c2663\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " Oct 30 19:43:40.856632 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.856564 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-metrics-certs\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-metrics-certs\") pod \"613f4cd0-8edb-4fbc-9629-d40b926c2663\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " Oct 30 19:43:40.856632 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.856622 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-maas-api-tls\") pod \"613f4cd0-8edb-4fbc-9629-d40b926c2663\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " Oct 30 19:43:40.856736 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.856663 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/613f4cd0-8edb-4fbc-9629-d40b926c2663-kube-api-access-nmncf\" (UniqueName: \"kubernetes.io/projected/613f4cd0-8edb-4fbc-9629-d40b926c2663-kube-api-access-nmncf\") pod \"613f4cd0-8edb-4fbc-9629-d40b926c2663\" (UID: \"613f4cd0-8edb-4fbc-9629-d40b926c2663\") " Oct 30 19:43:40.856876 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.856847 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/613f4cd0-8edb-4fbc-9629-d40b926c2663-openshift-service-ca" pod "613f4cd0-8edb-4fbc-9629-d40b926c2663" (UID: "613f4cd0-8edb-4fbc-9629-d40b926c2663"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:43:40.856992 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.856972 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/613f4cd0-8edb-4fbc-9629-d40b926c2663-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:43:40.858882 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.858858 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-maas-api-tls" pod "613f4cd0-8edb-4fbc-9629-d40b926c2663" (UID: "613f4cd0-8edb-4fbc-9629-d40b926c2663"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:43:40.859106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.859086 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/613f4cd0-8edb-4fbc-9629-d40b926c2663-kube-api-access-nmncf" pod "613f4cd0-8edb-4fbc-9629-d40b926c2663" (UID: "613f4cd0-8edb-4fbc-9629-d40b926c2663"). InnerVolumeSpecName "kube-api-access-nmncf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:43:40.859206 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.859142 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-metrics-certs" pod "613f4cd0-8edb-4fbc-9629-d40b926c2663" (UID: "613f4cd0-8edb-4fbc-9629-d40b926c2663"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:43:40.888393 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.888356 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:40.957552 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.957480 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:43:40.957552 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.957504 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/613f4cd0-8edb-4fbc-9629-d40b926c2663-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:43:40.957552 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:40.957515 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmncf\" (UniqueName: \"kubernetes.io/projected/613f4cd0-8edb-4fbc-9629-d40b926c2663-kube-api-access-nmncf\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:43:41.377590 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.377526 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:41.599200 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.599167 5627 generic.go:356] "Generic (PLEG): container finished" podID="613f4cd0-8edb-4fbc-9629-d40b926c2663" containerID="258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb" exitCode=0 Oct 30 19:43:41.599627 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.599220 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" Oct 30 19:43:41.599627 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.599257 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" event={"ID":"613f4cd0-8edb-4fbc-9629-d40b926c2663","Type":"ContainerDied","Data":"258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb"} Oct 30 19:43:41.599627 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.599328 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74" event={"ID":"613f4cd0-8edb-4fbc-9629-d40b926c2663","Type":"ContainerDied","Data":"4b76faf113d526e37143d24419a08f8de88df8f8fdc9ded7a707b833d264b5d0"} Oct 30 19:43:41.599627 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.599352 5627 scope.go:122] "RemoveContainer" containerID="258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb" Oct 30 19:43:41.608471 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.608455 5627 scope.go:122] "RemoveContainer" containerID="258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb" Oct 30 19:43:41.608773 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:43:41.608754 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb\": container with ID starting with 258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb not found: ID does not exist" containerID="258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb" Oct 30 19:43:41.608829 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.608783 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb"} err="failed to get container status \"258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb\": rpc error: code = NotFound desc = could not find container \"258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb\": container with ID starting with 258caa46f49b728261e9626b4f21d88f0c88e4c821c59ba988915fbf6dd96bfb not found: ID does not exist" Oct 30 19:43:41.618983 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.618960 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74"] Oct 30 19:43:41.626996 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:41.626976 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-ad6b27-6d84577544-hdb74"] Oct 30 19:43:43.396957 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:43.396924 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="613f4cd0-8edb-4fbc-9629-d40b926c2663" path="/var/lib/kubelet/pods/613f4cd0-8edb-4fbc-9629-d40b926c2663/volumes" Oct 30 19:43:44.025542 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:44.025502 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" podUID="3b27c6d4-f707-4af1-a435-d42b5108cb9d" containerName="payload-processing" containerID="cri-o://d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0" gracePeriod=25 Oct 30 19:43:44.294745 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:44.294643 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" podUID="3b27c6d4-f707-4af1-a435-d42b5108cb9d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:43:46.427540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:46.427507 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:48.448990 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:48.448948 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:49.737130 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:49.737100 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:52.085260 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.085155 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:52.716883 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.716845 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x"] Oct 30 19:43:52.717353 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.717335 5627 cpu_manager.go:414] "containerMap: removing container" podUID="613f4cd0-8edb-4fbc-9629-d40b926c2663" containerName="maas-api" Oct 30 19:43:52.717426 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.717354 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="613f4cd0-8edb-4fbc-9629-d40b926c2663" containerName="maas-api" Oct 30 19:43:52.717491 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.717438 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="613f4cd0-8edb-4fbc-9629-d40b926c2663" containerName="maas-api" Oct 30 19:43:52.721638 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.721619 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.733108 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.733086 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-56e018f0-kserve-self-signed-certs\"" Oct 30 19:43:52.736664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.736637 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x"] Oct 30 19:43:52.768514 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.768494 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-dshm\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.768618 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.768532 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-home\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.768618 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.768554 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-tmp-dir\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.768693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.768610 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-model-cache\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.768693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.768642 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggbrb\" (UniqueName: \"kubernetes.io/projected/0825ca9a-6e1e-4b16-901a-368d889af223-kube-api-access-ggbrb\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.768775 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.768759 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0825ca9a-6e1e-4b16-901a-368d889af223-tls-certs\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.783714 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.783682 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:43:52.869551 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.869517 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-home\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.869756 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.869560 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-tmp-dir\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.869756 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.869583 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-model-cache\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.869756 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.869608 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggbrb\" (UniqueName: \"kubernetes.io/projected/0825ca9a-6e1e-4b16-901a-368d889af223-kube-api-access-ggbrb\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.869756 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.869682 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0825ca9a-6e1e-4b16-901a-368d889af223-tls-certs\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.869991 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.869778 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-dshm\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.870045 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.870025 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-home\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.870096 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.870047 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-tmp-dir\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.870347 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.870306 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-model-cache\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.872136 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.872116 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-dshm\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.872589 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.872567 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0825ca9a-6e1e-4b16-901a-368d889af223-tls-certs\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:52.880306 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:52.880284 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggbrb\" (UniqueName: \"kubernetes.io/projected/0825ca9a-6e1e-4b16-901a-368d889af223-kube-api-access-ggbrb\") pod \"e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:53.033980 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:53.033954 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:53.316050 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:53.316019 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x"] Oct 30 19:43:53.317144 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:43:53.317121 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0825ca9a_6e1e_4b16_901a_368d889af223.slice/crio-b07987e70a9bee150a69b33539e7620daf6961131bbb5e619d9bf4e935491da7 WatchSource:0}: Error finding container b07987e70a9bee150a69b33539e7620daf6961131bbb5e619d9bf4e935491da7: Status 404 returned error can't find the container with id b07987e70a9bee150a69b33539e7620daf6961131bbb5e619d9bf4e935491da7 Oct 30 19:43:53.654543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:53.654452 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" event={"ID":"0825ca9a-6e1e-4b16-901a-368d889af223","Type":"ContainerStarted","Data":"bde32a1cec3c4638fe5ceda3461b245c74587be417f8529a348eb16e5dd79a62"} Oct 30 19:43:53.654543 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:53.654498 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" event={"ID":"0825ca9a-6e1e-4b16-901a-368d889af223","Type":"ContainerStarted","Data":"b07987e70a9bee150a69b33539e7620daf6961131bbb5e619d9bf4e935491da7"} Oct 30 19:43:53.683012 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:53.682963 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" podStartSLOduration=1.682949649 podStartE2EDuration="1.682949649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:53.680105371 +0000 UTC m=+1454.406922668" watchObservedRunningTime="2026-10-30 19:43:53.682949649 +0000 UTC m=+1454.409766946" Oct 30 19:43:54.034518 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:54.034492 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:43:54.294628 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:43:54.294532 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" podUID="3b27c6d4-f707-4af1-a435-d42b5108cb9d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:44:03.034122 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:03.034091 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:44:03.046896 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:03.046865 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:44:03.731915 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:03.731888 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:44:04.277538 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.277514 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:44:04.390179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.390099 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b27c6d4-f707-4af1-a435-d42b5108cb9d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b27c6d4-f707-4af1-a435-d42b5108cb9d-plugins-config-volume\") pod \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\" (UID: \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\") " Oct 30 19:44:04.390179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.390150 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b27c6d4-f707-4af1-a435-d42b5108cb9d-kube-api-access-zrssw\" (UniqueName: \"kubernetes.io/projected/3b27c6d4-f707-4af1-a435-d42b5108cb9d-kube-api-access-zrssw\") pod \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\" (UID: \"3b27c6d4-f707-4af1-a435-d42b5108cb9d\") " Oct 30 19:44:04.390514 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.390491 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b27c6d4-f707-4af1-a435-d42b5108cb9d-plugins-config-volume" pod "3b27c6d4-f707-4af1-a435-d42b5108cb9d" (UID: "3b27c6d4-f707-4af1-a435-d42b5108cb9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:44:04.392432 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.392406 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b27c6d4-f707-4af1-a435-d42b5108cb9d-kube-api-access-zrssw" pod "3b27c6d4-f707-4af1-a435-d42b5108cb9d" (UID: "3b27c6d4-f707-4af1-a435-d42b5108cb9d"). InnerVolumeSpecName "kube-api-access-zrssw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:44:04.491152 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.491123 5627 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b27c6d4-f707-4af1-a435-d42b5108cb9d-plugins-config-volume\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:04.491152 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.491149 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrssw\" (UniqueName: \"kubernetes.io/projected/3b27c6d4-f707-4af1-a435-d42b5108cb9d-kube-api-access-zrssw\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:04.733215 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.733128 5627 generic.go:356] "Generic (PLEG): container finished" podID="3b27c6d4-f707-4af1-a435-d42b5108cb9d" containerID="d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0" exitCode=0 Oct 30 19:44:04.733215 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.733188 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" Oct 30 19:44:04.733391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.733215 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" event={"ID":"3b27c6d4-f707-4af1-a435-d42b5108cb9d","Type":"ContainerDied","Data":"d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0"} Oct 30 19:44:04.733391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.733291 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf" event={"ID":"3b27c6d4-f707-4af1-a435-d42b5108cb9d","Type":"ContainerDied","Data":"06fa49333fab85119e157025831d72c358a600f67c67062110f1bc649631b44a"} Oct 30 19:44:04.733391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.733313 5627 scope.go:122] "RemoveContainer" containerID="d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0" Oct 30 19:44:04.742024 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.742010 5627 scope.go:122] "RemoveContainer" containerID="d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0" Oct 30 19:44:04.742268 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:44:04.742251 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0\": container with ID starting with d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0 not found: ID does not exist" containerID="d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0" Oct 30 19:44:04.742318 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.742275 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0"} err="failed to get container status \"d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0\": rpc error: code = NotFound desc = could not find container \"d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0\": container with ID starting with d422ebfb47a7f4147ec05221bfc5f11f68b446ee6a812491cedada548c3c05a0 not found: ID does not exist" Oct 30 19:44:04.767252 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.767231 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf"] Oct 30 19:44:04.791355 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:04.791334 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-ad6b27-64b9f8767m4hxf"] Oct 30 19:44:05.397242 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:05.397210 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b27c6d4-f707-4af1-a435-d42b5108cb9d" path="/var/lib/kubelet/pods/3b27c6d4-f707-4af1-a435-d42b5108cb9d/volumes" Oct 30 19:44:10.404047 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:44:10.404015 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:44:11.044203 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:11.044172 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:44:12.164047 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:12.164016 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:44:12.339474 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:12.339439 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:44:13.604691 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:13.604641 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:44:16.386836 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:16.386797 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-dlrbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:44:16.387416 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:16.387386 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dlrbh" podUID="177c6297-5b36-4a86-9baf-21793a0ec95d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:44:16.445819 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:16.445787 5627 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-n5rx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:44:16.446106 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:16.446078 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n5rx6" podUID="78d8bc24-bd69-4497-b7f8-cc98fcf3c379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:44:18.070925 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:18.070889 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:44:37.816840 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:37.816799 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x"] Oct 30 19:44:38.067793 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:38.067686 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:44:39.127314 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:39.127282 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:44:39.552765 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:39.552737 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:44:39.553916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:39.553893 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:44:40.409059 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:44:40.408989 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:44:46.588809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:46.588778 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:44:52.836916 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:52.836855 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" podUID="0825ca9a-6e1e-4b16-901a-368d889af223" containerName="main" containerID="cri-o://bde32a1cec3c4638fe5ceda3461b245c74587be417f8529a348eb16e5dd79a62" gracePeriod=45 Oct 30 19:44:53.039873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.039843 5627 generic.go:356] "Generic (PLEG): container finished" podID="0825ca9a-6e1e-4b16-901a-368d889af223" containerID="bde32a1cec3c4638fe5ceda3461b245c74587be417f8529a348eb16e5dd79a62" exitCode=0 Oct 30 19:44:53.040025 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.039898 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" event={"ID":"0825ca9a-6e1e-4b16-901a-368d889af223","Type":"ContainerDied","Data":"bde32a1cec3c4638fe5ceda3461b245c74587be417f8529a348eb16e5dd79a62"} Oct 30 19:44:53.085329 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.085308 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:44:53.140478 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.140415 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-dshm\") pod \"0825ca9a-6e1e-4b16-901a-368d889af223\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " Oct 30 19:44:53.140609 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.140477 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0825ca9a-6e1e-4b16-901a-368d889af223-kube-api-access-ggbrb\" (UniqueName: \"kubernetes.io/projected/0825ca9a-6e1e-4b16-901a-368d889af223-kube-api-access-ggbrb\") pod \"0825ca9a-6e1e-4b16-901a-368d889af223\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " Oct 30 19:44:53.140609 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.140522 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-tmp-dir\") pod \"0825ca9a-6e1e-4b16-901a-368d889af223\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " Oct 30 19:44:53.140609 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.140580 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-home\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-home\") pod \"0825ca9a-6e1e-4b16-901a-368d889af223\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " Oct 30 19:44:53.140815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.140639 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0825ca9a-6e1e-4b16-901a-368d889af223-tls-certs\" (UniqueName: \"kubernetes.io/secret/0825ca9a-6e1e-4b16-901a-368d889af223-tls-certs\") pod \"0825ca9a-6e1e-4b16-901a-368d889af223\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " Oct 30 19:44:53.140815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.140742 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-model-cache\") pod \"0825ca9a-6e1e-4b16-901a-368d889af223\" (UID: \"0825ca9a-6e1e-4b16-901a-368d889af223\") " Oct 30 19:44:53.140815 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.140783 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-tmp-dir" pod "0825ca9a-6e1e-4b16-901a-368d889af223" (UID: "0825ca9a-6e1e-4b16-901a-368d889af223"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:44:53.140959 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.140899 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-home" pod "0825ca9a-6e1e-4b16-901a-368d889af223" (UID: "0825ca9a-6e1e-4b16-901a-368d889af223"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:44:53.141025 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.141002 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-model-cache" pod "0825ca9a-6e1e-4b16-901a-368d889af223" (UID: "0825ca9a-6e1e-4b16-901a-368d889af223"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:44:53.141111 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.141096 5627 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-tmp-dir\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:53.141164 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.141113 5627 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-home\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:53.141164 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.141121 5627 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-model-cache\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:53.142922 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.142899 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-dshm" pod "0825ca9a-6e1e-4b16-901a-368d889af223" (UID: "0825ca9a-6e1e-4b16-901a-368d889af223"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:44:53.142922 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.142905 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0825ca9a-6e1e-4b16-901a-368d889af223-tls-certs" pod "0825ca9a-6e1e-4b16-901a-368d889af223" (UID: "0825ca9a-6e1e-4b16-901a-368d889af223"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:44:53.143058 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.142970 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0825ca9a-6e1e-4b16-901a-368d889af223-kube-api-access-ggbrb" pod "0825ca9a-6e1e-4b16-901a-368d889af223" (UID: "0825ca9a-6e1e-4b16-901a-368d889af223"). InnerVolumeSpecName "kube-api-access-ggbrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:44:53.242044 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.242014 5627 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0825ca9a-6e1e-4b16-901a-368d889af223-dshm\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:53.242044 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.242039 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggbrb\" (UniqueName: \"kubernetes.io/projected/0825ca9a-6e1e-4b16-901a-368d889af223-kube-api-access-ggbrb\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:53.242196 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:53.242061 5627 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0825ca9a-6e1e-4b16-901a-368d889af223-tls-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:54.084782 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:54.084741 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" event={"ID":"0825ca9a-6e1e-4b16-901a-368d889af223","Type":"ContainerDied","Data":"b07987e70a9bee150a69b33539e7620daf6961131bbb5e619d9bf4e935491da7"} Oct 30 19:44:54.085351 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:54.085330 5627 scope.go:122] "RemoveContainer" containerID="bde32a1cec3c4638fe5ceda3461b245c74587be417f8529a348eb16e5dd79a62" Oct 30 19:44:54.085639 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:54.085619 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x" Oct 30 19:44:54.113105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:54.113083 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x"] Oct 30 19:44:54.142727 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:54.135020 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-56e018f0-kserve-5965f54f89-rjq2x"] Oct 30 19:44:55.399963 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:44:55.399929 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0825ca9a-6e1e-4b16-901a-368d889af223" path="/var/lib/kubelet/pods/0825ca9a-6e1e-4b16-901a-368d889af223/volumes" Oct 30 19:45:10.402984 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:45:10.402947 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:45:15.249587 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.249556 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt"] Oct 30 19:45:15.250020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.249952 5627 cpu_manager.go:414] "containerMap: removing container" podUID="0825ca9a-6e1e-4b16-901a-368d889af223" containerName="main" Oct 30 19:45:15.250020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.249965 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0825ca9a-6e1e-4b16-901a-368d889af223" containerName="main" Oct 30 19:45:15.250020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.249983 5627 cpu_manager.go:414] "containerMap: removing container" podUID="3b27c6d4-f707-4af1-a435-d42b5108cb9d" containerName="payload-processing" Oct 30 19:45:15.250020 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.249988 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b27c6d4-f707-4af1-a435-d42b5108cb9d" containerName="payload-processing" Oct 30 19:45:15.250177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.250046 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="0825ca9a-6e1e-4b16-901a-368d889af223" containerName="main" Oct 30 19:45:15.250177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.250057 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b27c6d4-f707-4af1-a435-d42b5108cb9d" containerName="payload-processing" Oct 30 19:45:15.253023 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.253008 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.269578 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.269553 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt"] Oct 30 19:45:15.413275 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.413245 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4ggr\" (UniqueName: \"kubernetes.io/projected/96e8227c-b7ae-45e7-b5bc-01562a8af474-kube-api-access-z4ggr\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.413275 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.413280 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96e8227c-b7ae-45e7-b5bc-01562a8af474-openshift-service-ca\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.413495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.413406 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96e8227c-b7ae-45e7-b5bc-01562a8af474-maas-api-tls\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.413495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.413439 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96e8227c-b7ae-45e7-b5bc-01562a8af474-metrics-certs\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.514425 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.514345 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96e8227c-b7ae-45e7-b5bc-01562a8af474-maas-api-tls\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.514425 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.514386 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96e8227c-b7ae-45e7-b5bc-01562a8af474-metrics-certs\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.514651 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.514465 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4ggr\" (UniqueName: \"kubernetes.io/projected/96e8227c-b7ae-45e7-b5bc-01562a8af474-kube-api-access-z4ggr\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.514651 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.514495 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96e8227c-b7ae-45e7-b5bc-01562a8af474-openshift-service-ca\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.515176 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.515154 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96e8227c-b7ae-45e7-b5bc-01562a8af474-openshift-service-ca\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.517006 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.516985 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96e8227c-b7ae-45e7-b5bc-01562a8af474-metrics-certs\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.517098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.517034 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96e8227c-b7ae-45e7-b5bc-01562a8af474-maas-api-tls\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.527462 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.527433 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4ggr\" (UniqueName: \"kubernetes.io/projected/96e8227c-b7ae-45e7-b5bc-01562a8af474-kube-api-access-z4ggr\") pod \"maas-api-5df7c6c68c-5zkbt\" (UID: \"96e8227c-b7ae-45e7-b5bc-01562a8af474\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.565976 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.565956 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:15.859197 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.859167 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt"] Oct 30 19:45:15.860931 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:45:15.860906 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96e8227c_b7ae_45e7_b5bc_01562a8af474.slice/crio-f8efa7ab064e7c1c3d094d3add54ac2e2ef53d25c3700157c5aaaa433f0d3d7d WatchSource:0}: Error finding container f8efa7ab064e7c1c3d094d3add54ac2e2ef53d25c3700157c5aaaa433f0d3d7d: Status 404 returned error can't find the container with id f8efa7ab064e7c1c3d094d3add54ac2e2ef53d25c3700157c5aaaa433f0d3d7d Oct 30 19:45:15.862389 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:15.862374 5627 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:45:16.180892 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:16.180806 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" event={"ID":"96e8227c-b7ae-45e7-b5bc-01562a8af474","Type":"ContainerStarted","Data":"f8efa7ab064e7c1c3d094d3add54ac2e2ef53d25c3700157c5aaaa433f0d3d7d"} Oct 30 19:45:17.187168 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:17.187137 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" event={"ID":"96e8227c-b7ae-45e7-b5bc-01562a8af474","Type":"ContainerStarted","Data":"199fd53565ec7294383aa9ee3bc88318fbc10316a4dde6f5260d75b125a89c17"} Oct 30 19:45:17.187565 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:17.187276 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:17.214368 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:17.214328 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" podStartSLOduration=1.836911907 podStartE2EDuration="2.214316185s" totalImagesPullingTime="377.404278ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:17.213226087 +0000 UTC m=+1537.940043384" watchObservedRunningTime="2026-10-30 19:45:17.214316185 +0000 UTC m=+1537.941133481" Oct 30 19:45:22.262337 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:22.262257 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:22.895798 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:22.895760 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:23.102307 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:23.102269 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:23.197325 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:23.197265 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-5zkbt" Oct 30 19:45:25.335231 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:25.335193 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:25.486063 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:25.486032 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:26.806787 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:26.806753 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:27.047998 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:27.047967 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:27.238251 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:27.238172 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:27.388378 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:27.388343 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:28.358369 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:28.358333 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:29.214908 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:29.214877 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:30.438819 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:30.438782 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:31.131437 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:31.131404 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9"] Oct 30 19:45:31.131682 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:31.131656 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" podUID="a3bbc926-da38-4920-99fc-0d073a24f109" containerName="maas-api" containerID="cri-o://e6cfe60dfdf973ff3e7848440e24e9413ec6a67bedcbaedb0daa98a56589057e" gracePeriod=30 Oct 30 19:45:31.514592 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:31.514550 5627 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" podUID="a3bbc926-da38-4920-99fc-0d073a24f109" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.26:8443/health\": dial tcp 10.133.0.26:8443: connect: connection refused" Oct 30 19:45:31.537294 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:31.537260 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:32.256967 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.256943 5627 generic.go:356] "Generic (PLEG): container finished" podID="a3bbc926-da38-4920-99fc-0d073a24f109" containerID="e6cfe60dfdf973ff3e7848440e24e9413ec6a67bedcbaedb0daa98a56589057e" exitCode=0 Oct 30 19:45:32.257059 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.257016 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" event={"ID":"a3bbc926-da38-4920-99fc-0d073a24f109","Type":"ContainerDied","Data":"e6cfe60dfdf973ff3e7848440e24e9413ec6a67bedcbaedb0daa98a56589057e"} Oct 30 19:45:32.378291 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.378268 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:45:32.437347 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.437315 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:32.462284 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.462208 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-metrics-certs\") pod \"a3bbc926-da38-4920-99fc-0d073a24f109\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " Oct 30 19:45:32.462284 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.462270 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3bbc926-da38-4920-99fc-0d073a24f109-kube-api-access-r8m8m\" (UniqueName: \"kubernetes.io/projected/a3bbc926-da38-4920-99fc-0d073a24f109-kube-api-access-r8m8m\") pod \"a3bbc926-da38-4920-99fc-0d073a24f109\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " Oct 30 19:45:32.462458 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.462352 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3bbc926-da38-4920-99fc-0d073a24f109-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbc926-da38-4920-99fc-0d073a24f109-openshift-service-ca\") pod \"a3bbc926-da38-4920-99fc-0d073a24f109\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " Oct 30 19:45:32.462458 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.462405 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-maas-api-tls\") pod \"a3bbc926-da38-4920-99fc-0d073a24f109\" (UID: \"a3bbc926-da38-4920-99fc-0d073a24f109\") " Oct 30 19:45:32.463178 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.462759 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3bbc926-da38-4920-99fc-0d073a24f109-openshift-service-ca" pod "a3bbc926-da38-4920-99fc-0d073a24f109" (UID: "a3bbc926-da38-4920-99fc-0d073a24f109"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:45:32.463178 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.462865 5627 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbc926-da38-4920-99fc-0d073a24f109-openshift-service-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:45:32.464497 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.464476 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-metrics-certs" pod "a3bbc926-da38-4920-99fc-0d073a24f109" (UID: "a3bbc926-da38-4920-99fc-0d073a24f109"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:45:32.464674 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.464653 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-maas-api-tls" pod "a3bbc926-da38-4920-99fc-0d073a24f109" (UID: "a3bbc926-da38-4920-99fc-0d073a24f109"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:45:32.464763 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.464716 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3bbc926-da38-4920-99fc-0d073a24f109-kube-api-access-r8m8m" pod "a3bbc926-da38-4920-99fc-0d073a24f109" (UID: "a3bbc926-da38-4920-99fc-0d073a24f109"). InnerVolumeSpecName "kube-api-access-r8m8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:45:32.563562 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.563535 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-maas-api-tls\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:45:32.563562 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.563557 5627 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbc926-da38-4920-99fc-0d073a24f109-metrics-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:45:32.563935 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:32.563568 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8m8m\" (UniqueName: \"kubernetes.io/projected/a3bbc926-da38-4920-99fc-0d073a24f109-kube-api-access-r8m8m\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:45:33.263362 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:33.263324 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" event={"ID":"a3bbc926-da38-4920-99fc-0d073a24f109","Type":"ContainerDied","Data":"5edf56de8f84cde8efe800f8240c481c1ee0912d9b2b92ab6bee5ad674eb19e4"} Oct 30 19:45:33.263555 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:33.263368 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9" Oct 30 19:45:33.263555 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:33.263416 5627 scope.go:122] "RemoveContainer" containerID="e6cfe60dfdf973ff3e7848440e24e9413ec6a67bedcbaedb0daa98a56589057e" Oct 30 19:45:33.287023 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:33.286998 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9"] Oct 30 19:45:33.292119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:33.292097 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-788b7ddf75-lfwk9"] Oct 30 19:45:33.398067 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:33.398039 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3bbc926-da38-4920-99fc-0d073a24f109" path="/var/lib/kubelet/pods/a3bbc926-da38-4920-99fc-0d073a24f109/volumes" Oct 30 19:45:33.671978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:33.671890 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:34.689519 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:34.689478 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:35.559622 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:35.559593 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:36.581083 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:36.581051 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:37.433805 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:37.433772 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:38.217675 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:38.217642 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:39.192022 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:39.191989 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:39.894305 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:39.894264 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:40.405174 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:45:40.405140 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:45:40.757988 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:40.757952 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:41.271664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.271631 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2"] Oct 30 19:45:41.272094 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.272016 5627 cpu_manager.go:414] "containerMap: removing container" podUID="a3bbc926-da38-4920-99fc-0d073a24f109" containerName="maas-api" Oct 30 19:45:41.272094 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.272031 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3bbc926-da38-4920-99fc-0d073a24f109" containerName="maas-api" Oct 30 19:45:41.272094 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.272094 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3bbc926-da38-4920-99fc-0d073a24f109" containerName="maas-api" Oct 30 19:45:41.276626 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.276604 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.281803 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.281781 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 19:45:41.283250 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.283229 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\"" Oct 30 19:45:41.297459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.297437 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2"] Oct 30 19:45:41.350877 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.350838 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.351113 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.350942 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb9s5\" (UniqueName: \"kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.351113 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.351031 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.452380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.452348 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.452583 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.452560 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb9s5\" (UniqueName: \"kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.452680 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.452668 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.453108 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.453091 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.455178 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.455160 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:41.462306 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:45:41.462286 5627 projected.go:196] Error preparing data for projected volume kube-api-access-lb9s5 for pod odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2: failed to fetch token: serviceaccounts "maas-api-cleanup" not found Oct 30 19:45:41.462399 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:45:41.462377 5627 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5 podName:bfcb8891-93dc-428c-8325-6360a7c05e12 nodeName:}" failed. No retries permitted until 2026-10-30 19:45:41.962361848 +0000 UTC m=+1562.689179136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lb9s5" (UniqueName: "kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5") pod "maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" (UID: "bfcb8891-93dc-428c-8325-6360a7c05e12") : failed to fetch token: serviceaccounts "maas-api-cleanup" not found Oct 30 19:45:41.777666 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:41.777630 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:42.057812 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:42.057736 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb9s5\" (UniqueName: \"kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:42.067585 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:42.067561 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb9s5\" (UniqueName: \"kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5\") pod \"maas-api-revoke-keys-e2e-ait-77ab1071-gttr2\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:42.189380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:42.189345 5627 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" secret="" err="secret \"maas-api-cleanup-dockercfg-4kw2n\" not found" Oct 30 19:45:42.189535 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:42.189420 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:42.467262 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:42.467215 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2"] Oct 30 19:45:42.469581 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:45:42.469553 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfcb8891_93dc_428c_8325_6360a7c05e12.slice/crio-8edd3530a394c9d54a50368b330eee0ce4852313acea8a318194d7c8ea229d04 WatchSource:0}: Error finding container 8edd3530a394c9d54a50368b330eee0ce4852313acea8a318194d7c8ea229d04: Status 404 returned error can't find the container with id 8edd3530a394c9d54a50368b330eee0ce4852313acea8a318194d7c8ea229d04 Oct 30 19:45:42.652035 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:42.651982 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:43.308602 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:43.308568 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" event={"ID":"bfcb8891-93dc-428c-8325-6360a7c05e12","Type":"ContainerStarted","Data":"8edd3530a394c9d54a50368b330eee0ce4852313acea8a318194d7c8ea229d04"} Oct 30 19:45:43.627578 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:43.627494 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:44.466255 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:44.466220 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:45.321731 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:45.321594 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" event={"ID":"bfcb8891-93dc-428c-8325-6360a7c05e12","Type":"ContainerStarted","Data":"43a743be01139adfbb981679f33e0934a981351e395e3d5c44058cdf1873c79b"} Oct 30 19:45:45.322153 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:45.321916 5627 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" secret="" err="secret \"maas-api-cleanup-dockercfg-4kw2n\" not found" Oct 30 19:45:45.349791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:45.349746 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" podStartSLOduration=2.118562649 podStartE2EDuration="4.349728774s" totalImagesPullingTime="2.231166125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:45.349335019 +0000 UTC m=+1566.076152321" watchObservedRunningTime="2026-10-30 19:45:45.349728774 +0000 UTC m=+1566.076546070" Oct 30 19:45:45.376917 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:45.376889 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:46.240450 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:46.240420 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:46.326925 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:46.326892 5627 generic.go:356] "Generic (PLEG): container finished" podID="bfcb8891-93dc-428c-8325-6360a7c05e12" containerID="43a743be01139adfbb981679f33e0934a981351e395e3d5c44058cdf1873c79b" exitCode=0 Oct 30 19:45:46.327333 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:46.326980 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" event={"ID":"bfcb8891-93dc-428c-8325-6360a7c05e12","Type":"ContainerDied","Data":"43a743be01139adfbb981679f33e0934a981351e395e3d5c44058cdf1873c79b"} Oct 30 19:45:46.535273 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:46.535197 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:47.109484 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.109453 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:47.463848 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.463823 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:47.610822 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.610786 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5\" (UniqueName: \"kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5\") pod \"bfcb8891-93dc-428c-8325-6360a7c05e12\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " Oct 30 19:45:47.611014 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.610839 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-xks-ca\") pod \"bfcb8891-93dc-428c-8325-6360a7c05e12\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " Oct 30 19:45:47.611014 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.610957 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-openshift-ca\") pod \"bfcb8891-93dc-428c-8325-6360a7c05e12\" (UID: \"bfcb8891-93dc-428c-8325-6360a7c05e12\") " Oct 30 19:45:47.611307 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.611283 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-openshift-ca" pod "bfcb8891-93dc-428c-8325-6360a7c05e12" (UID: "bfcb8891-93dc-428c-8325-6360a7c05e12"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:45:47.613070 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.613048 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-xks-ca" pod "bfcb8891-93dc-428c-8325-6360a7c05e12" (UID: "bfcb8891-93dc-428c-8325-6360a7c05e12"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:45:47.613155 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.613137 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5" pod "bfcb8891-93dc-428c-8325-6360a7c05e12" (UID: "bfcb8891-93dc-428c-8325-6360a7c05e12"). InnerVolumeSpecName "kube-api-access-lb9s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:45:47.712459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.712372 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb9s5\" (UniqueName: \"kubernetes.io/projected/bfcb8891-93dc-428c-8325-6360a7c05e12-kube-api-access-lb9s5\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:45:47.712459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.712410 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-xks-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:45:47.712459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:47.712424 5627 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bfcb8891-93dc-428c-8325-6360a7c05e12-maas-api-openshift-ca\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:45:48.055777 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:48.055743 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:48.337153 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:48.337073 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" Oct 30 19:45:48.337153 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:48.337092 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2" event={"ID":"bfcb8891-93dc-428c-8325-6360a7c05e12","Type":"ContainerDied","Data":"8edd3530a394c9d54a50368b330eee0ce4852313acea8a318194d7c8ea229d04"} Oct 30 19:45:48.337153 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:48.337139 5627 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8edd3530a394c9d54a50368b330eee0ce4852313acea8a318194d7c8ea229d04" Oct 30 19:45:48.976490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:48.976460 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:49.691468 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:49.691431 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:50.516175 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:50.516144 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:51.211882 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:51.211846 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:51.574717 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:51.574673 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:51.757011 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:51.756976 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:52.130105 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:52.130065 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:52.833213 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:52.833180 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:52.998046 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:52.998013 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:53.035126 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:53.035100 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2"] Oct 30 19:45:53.039049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:53.039028 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77ab1071-gttr2"] Oct 30 19:45:53.161109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:53.161021 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:53.396509 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:53.396477 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfcb8891-93dc-428c-8325-6360a7c05e12" path="/var/lib/kubelet/pods/bfcb8891-93dc-428c-8325-6360a7c05e12/volumes" Oct 30 19:45:57.765413 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:57.765377 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:59.064065 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.064035 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:59.473457 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.473388 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx"] Oct 30 19:45:59.473794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.473779 5627 cpu_manager.go:414] "containerMap: removing container" podUID="bfcb8891-93dc-428c-8325-6360a7c05e12" containerName="revoke-keys" Oct 30 19:45:59.473850 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.473796 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfcb8891-93dc-428c-8325-6360a7c05e12" containerName="revoke-keys" Oct 30 19:45:59.473890 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.473854 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfcb8891-93dc-428c-8325-6360a7c05e12" containerName="revoke-keys" Oct 30 19:45:59.476863 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.476849 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.486326 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.486304 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-7ed3da-openshift-default-dockercfg-xxs2f\"" Oct 30 19:45:59.493303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.493272 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx"] Oct 30 19:45:59.620121 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.620087 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:45:59.628200 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628171 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-certs\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.628364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628212 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-token\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.628364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628267 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istiod-ca-cert\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.628364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628300 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-socket\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.628364 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628335 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-data\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.628534 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628428 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-credential-socket\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.628534 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628456 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-envoy\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.628534 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628489 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57lld\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-kube-api-access-57lld\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.628730 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.628683 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-podinfo\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729384 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57lld\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-kube-api-access-57lld\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729431 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-podinfo\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729459 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729452 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-certs\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729471 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-token\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729497 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istiod-ca-cert\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729530 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-socket\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729561 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-data\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729621 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-credential-socket\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.729749 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729657 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-envoy\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.730071 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729934 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-certs\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.730071 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.729953 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-data\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.730166 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.730099 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-credential-socket\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.730280 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.730257 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-socket\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.730369 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.730353 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istiod-ca-cert\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.731906 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.731885 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-envoy\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.732066 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.732046 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-podinfo\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.741185 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.741159 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-token\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.741185 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.741174 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57lld\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-kube-api-access-57lld\") pod \"e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.789276 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.789254 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:45:59.822418 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:45:59.822393 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:00.082369 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:00.082341 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx"] Oct 30 19:46:00.083360 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:46:00.083329 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42186c40_5e91_4467_8f25_6d5ad0fabdb8.slice/crio-36776d106f45d702ec33118a70a44460127d0d81c09966b55a735438bab10b5a WatchSource:0}: Error finding container 36776d106f45d702ec33118a70a44460127d0d81c09966b55a735438bab10b5a: Status 404 returned error can't find the container with id 36776d106f45d702ec33118a70a44460127d0d81c09966b55a735438bab10b5a Oct 30 19:46:00.085810 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:00.085774 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:46:00.085886 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:00.085857 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:46:00.085927 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:00.085899 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:46:00.391109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:00.391021 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" event={"ID":"42186c40-5e91-4467-8f25-6d5ad0fabdb8","Type":"ContainerStarted","Data":"80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2"} Oct 30 19:46:00.391109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:00.391083 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" event={"ID":"42186c40-5e91-4467-8f25-6d5ad0fabdb8","Type":"ContainerStarted","Data":"36776d106f45d702ec33118a70a44460127d0d81c09966b55a735438bab10b5a"} Oct 30 19:46:00.420507 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:00.420462 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" podStartSLOduration=1.420448964 podStartE2EDuration="1.420448964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:00.418613949 +0000 UTC m=+1581.145431248" watchObservedRunningTime="2026-10-30 19:46:00.420448964 +0000 UTC m=+1581.147266261" Oct 30 19:46:00.789447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:00.789409 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:01.794615 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:01.794586 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:02.400940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:02.400912 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:02.401779 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:02.401759 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:02.763794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:02.763763 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:03.008698 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:03.008666 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:04.098999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:04.098955 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:04.536429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:04.536399 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:05.122968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:05.122934 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:06.237182 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:06.237151 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:07.245190 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:07.245121 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:08.196127 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:08.196085 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:09.124737 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:09.123173 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:09.959119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:09.959066 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:10.402892 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:46:10.402855 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:46:10.722794 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:10.721371 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:11.644290 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:11.644249 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:12.552458 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:12.552427 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:13.476925 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:13.476895 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:14.176946 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:14.176909 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:14.897468 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:14.897431 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:15.709108 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:15.709069 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:16.467900 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:16.467863 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:16.492725 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:16.492660 5627 generic.go:356] "Generic (PLEG): container finished" podID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerID="80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2" exitCode=0 Oct 30 19:46:16.492927 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:16.492736 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" event={"ID":"42186c40-5e91-4467-8f25-6d5ad0fabdb8","Type":"ContainerDied","Data":"80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2"} Oct 30 19:46:16.493218 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:16.493194 5627 scope.go:122] "RemoveContainer" containerID="80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2" Oct 30 19:46:16.494685 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:16.494658 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:46:16.494809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:16.494748 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:46:16.494809 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:16.494788 5627 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"} Oct 30 19:46:17.254693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:17.254665 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:17.401693 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:17.401662 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:17.500031 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:17.499994 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" event={"ID":"42186c40-5e91-4467-8f25-6d5ad0fabdb8","Type":"ContainerStarted","Data":"25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca"} Oct 30 19:46:17.789688 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:17.789651 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:17.790955 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:17.790924 5627 patch_prober.go:29] interesting pod/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:46:17.791101 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:17.790982 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Oct 30 19:46:17.823448 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:17.819550 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:18.142787 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:18.141508 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:18.790365 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:18.790320 5627 patch_prober.go:29] interesting pod/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:46:18.790893 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:18.790389 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Oct 30 19:46:18.939843 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:18.939782 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:19.705119 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:19.705085 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:19.790031 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:19.789985 5627 patch_prober.go:29] interesting pod/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:46:19.790177 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:19.790054 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Oct 30 19:46:20.615966 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:20.615925 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:20.790335 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:20.790300 5627 patch_prober.go:29] interesting pod/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:46:20.790508 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:20.790368 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Oct 30 19:46:21.454924 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:21.454885 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:21.790028 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:21.789993 5627 patch_prober.go:29] interesting pod/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:46:21.790413 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:21.790045 5627 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Oct 30 19:46:22.244337 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:22.244306 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:22.793424 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:22.793397 5627 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:22.793921 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:22.793689 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:22.794510 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:22.794496 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:46:23.023141 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:23.023106 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:23.907885 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:23.907846 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:24.607484 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:24.607451 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:25.303349 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:25.303312 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:26.029867 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:26.029815 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:26.968408 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:26.968355 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:27.718224 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:27.718188 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:28.636070 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:28.636008 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:29.649715 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:29.649663 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:30.428806 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:30.428774 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:31.444110 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:31.444072 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:32.362218 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:32.362175 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:33.228621 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:33.228571 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:33.658729 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:33.658662 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:34.027949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:34.027917 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:34.873118 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:34.873072 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:35.667049 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:35.667006 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:36.500677 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:36.500644 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:37.243927 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:37.243895 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:38.018617 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:38.018572 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:38.812109 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:38.812074 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:39.500999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:39.500969 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:40.306670 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:40.306637 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:40.402964 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:46:40.402926 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:46:41.113270 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:41.113238 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:42.032194 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:42.032127 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:42.947588 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:42.947551 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:43.813678 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:43.813639 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:44.846074 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:44.846043 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:45.507593 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:45.507557 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:46.298131 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:46.298091 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:46.578026 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:46.577943 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:47.302400 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:47.302370 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:48.021344 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:48.021311 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:48.909238 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:48.909209 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:49.676813 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:49.676765 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:50.406797 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:50.406686 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:51.363394 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:51.363359 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:52.202954 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:52.202916 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:52.906717 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:52.906665 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:53.734427 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:53.734394 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:54.505843 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:54.505811 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:55.151368 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:55.151331 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:55.295451 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:55.295420 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:55.433795 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:55.433687 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:56.087423 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:56.087390 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:56.872070 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:56.872037 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:57.734044 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:57.734013 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:58.498995 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:58.498955 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:46:59.324605 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:46:59.324575 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:00.105878 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:00.105845 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:00.968187 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:00.968152 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:02.005887 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:02.005850 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:02.825581 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:02.825549 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:03.598914 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:03.598875 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:04.452717 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:04.452658 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:05.225226 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:05.225191 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:06.417871 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:06.417841 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:07.416479 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:07.416440 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:08.143544 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:08.143511 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:09.030404 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:09.030369 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:09.822988 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:09.822951 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:10.403539 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:47:10.403497 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:47:10.644873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:10.644794 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:11.506554 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:11.506521 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:12.235096 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:12.235056 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:13.024912 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:13.024880 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:13.809255 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:13.809225 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:14.670295 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:14.670263 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:15.409981 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:15.409948 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:16.454044 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:16.454010 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:17.168477 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:17.168443 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:18.043494 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:18.043454 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:18.827973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:18.827941 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:19.608524 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:19.608472 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:20.374715 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:20.374676 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:21.094454 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:21.094420 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:21.783679 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:21.783651 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:22.593460 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:22.593413 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:23.501338 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:23.501303 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:24.097150 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:24.097091 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:24.961107 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:24.961075 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:25.800885 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:25.800836 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:26.510058 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:26.510010 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:27.235576 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:27.235542 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:27.994392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:27.994356 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:28.733933 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:28.733894 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:29.514931 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:29.514898 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:30.309434 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:30.309394 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:31.123284 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:31.123240 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:31.877873 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:31.877845 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:32.611741 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:32.611696 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:33.356947 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:33.356912 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:34.104093 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:34.104056 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:34.921945 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:34.921911 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:35.607001 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:35.606967 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:36.361606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:36.361575 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:36.990484 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:36.990455 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:37.764490 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:37.764454 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:38.891624 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:38.891593 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:39.582620 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:39.582585 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:40.387485 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:40.387454 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:40.403563 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:47:40.403527 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:47:41.128954 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:41.128918 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:42.008532 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:42.008495 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:42.826179 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:42.826141 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:43.600985 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:43.600953 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:44.500369 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:44.500335 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:45.443949 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:45.443921 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:46.372372 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:46.372337 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:46.591688 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:46.591656 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:47.132036 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:47.132004 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:48.219886 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:48.219849 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:48.590129 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:48.590099 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:48.718264 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:48.718232 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:49.222383 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:49.222344 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:54.769067 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:54.769030 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:54.920655 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:54.920617 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:55.041455 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:55.041378 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:47:55.531564 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:47:55.531533 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:02.111987 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:02.111950 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:02.318856 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:02.318807 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:02.427515 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:02.427441 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx"] Oct 30 19:48:02.427664 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:02.427645 5627 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" containerID="cri-o://25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca" gracePeriod=30 Oct 30 19:48:02.540968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:02.540930 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:07.729694 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.729668 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:48:07.824601 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824560 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-data\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.824819 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824607 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-credential-socket\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.824819 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824666 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-podinfo\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.824819 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824742 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-envoy\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.824819 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824777 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istiod-ca-cert\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.824819 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824817 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-socket\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.825099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824844 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-certs\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.825099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824875 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-kube-api-access-57lld\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-kube-api-access-57lld\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.825099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824907 5627 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-token\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-token\") pod \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\" (UID: \"42186c40-5e91-4467-8f25-6d5ad0fabdb8\") " Oct 30 19:48:07.825099 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.824907 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-data" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:07.825359 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.825288 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istiod-ca-cert" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:48:07.825359 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.825297 5627 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-data\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.825742 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.825522 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-socket" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:07.825860 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.825754 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-certs" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:07.826003 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.825978 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-credential-socket" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:07.827948 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.827906 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-token" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:48:07.828230 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.828209 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-envoy" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:48:07.828327 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.828303 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-kube-api-access-57lld" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "kube-api-access-57lld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:48:07.828821 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.828801 5627 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-podinfo" pod "42186c40-5e91-4467-8f25-6d5ad0fabdb8" (UID: "42186c40-5e91-4467-8f25-6d5ad0fabdb8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:48:07.926241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.926068 5627 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.926241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.926111 5627 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-workload-certs\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.926241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.926127 5627 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57lld\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-kube-api-access-57lld\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.926241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.926145 5627 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-token\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.926241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.926159 5627 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-credential-socket\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.926241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.926173 5627 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-podinfo\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.926241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.926187 5627 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istio-envoy\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.926241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.926200 5627 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42186c40-5e91-4467-8f25-6d5ad0fabdb8-istiod-ca-cert\") on node \"ip-10-0-139-211.us-east-2.compute.internal\" DevicePath \"\"" Oct 30 19:48:07.997595 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.997563 5627 generic.go:356] "Generic (PLEG): container finished" podID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerID="25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca" exitCode=0 Oct 30 19:48:07.997826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.997605 5627 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" Oct 30 19:48:07.997826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.997640 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" event={"ID":"42186c40-5e91-4467-8f25-6d5ad0fabdb8","Type":"ContainerDied","Data":"25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca"} Oct 30 19:48:07.997826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.997698 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx" event={"ID":"42186c40-5e91-4467-8f25-6d5ad0fabdb8","Type":"ContainerDied","Data":"36776d106f45d702ec33118a70a44460127d0d81c09966b55a735438bab10b5a"} Oct 30 19:48:07.997826 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:07.997742 5627 scope.go:122] "RemoveContainer" containerID="25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca" Oct 30 19:48:08.010386 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:08.010364 5627 scope.go:122] "RemoveContainer" containerID="80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2" Oct 30 19:48:08.019540 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:08.019516 5627 scope.go:122] "RemoveContainer" containerID="25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca" Oct 30 19:48:08.019888 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:48:08.019868 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca\": container with ID starting with 25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca not found: ID does not exist" containerID="25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca" Oct 30 19:48:08.019951 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:08.019899 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca"} err="failed to get container status \"25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca\": rpc error: code = NotFound desc = could not find container \"25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca\": container with ID starting with 25c3a61ac014e6ce2fc3a4f3caff45077ca32d4f6cc2d607de2524e285477aca not found: ID does not exist" Oct 30 19:48:08.019951 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:08.019918 5627 scope.go:122] "RemoveContainer" containerID="80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2" Oct 30 19:48:08.020210 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:48:08.020189 5627 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2\": container with ID starting with 80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2 not found: ID does not exist" containerID="80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2" Oct 30 19:48:08.020303 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:08.020217 5627 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2"} err="failed to get container status \"80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2\": rpc error: code = NotFound desc = could not find container \"80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2\": container with ID starting with 80cb6463af9a669b6e235fbbb5baf3867ce09c9b23705b6785efd5791457c8f2 not found: ID does not exist" Oct 30 19:48:08.026970 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:08.026940 5627 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx"] Oct 30 19:48:08.032218 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:08.032195 5627 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7ed3da-openshift-default-64c5d5587c-725wx"] Oct 30 19:48:09.396960 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:09.396929 5627 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" path="/var/lib/kubelet/pods/42186c40-5e91-4467-8f25-6d5ad0fabdb8/volumes" Oct 30 19:48:10.407087 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:48:10.407030 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:48:13.937276 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:13.937242 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:29.809915 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:29.809857 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:29.939865 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:29.939834 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:40.402064 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:48:40.402028 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:48:46.642314 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:46.642279 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:48:52.542348 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:48:52.542312 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c87c8b988-knzjq_408b1443-a9e2-493d-9597-1e5512dd88ff/authorino/0.log" Oct 30 19:49:03.833296 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:03.833254 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-zl5gt_88809f9c-f262-4f69-9a56-4876ed7404da/manager/0.log" Oct 30 19:49:04.897847 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:04.897820 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4cvck_72b21b6b-0d07-45e9-8899-b625ff8d5c5e/proxy/0.log" Oct 30 19:49:05.648225 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:05.648192 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5df7c6c68c-5zkbt_96e8227c-b7ae-45e7-b5bc-01562a8af474/maas-api/0.log" Oct 30 19:49:08.636572 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:08.636539 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c87c8b988-knzjq_408b1443-a9e2-493d-9597-1e5512dd88ff/authorino/0.log" Oct 30 19:49:09.242532 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:09.242503 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-vs77r_d631061b-03d9-4e53-a42d-f1ce92fc2af4/kuadrant-console-plugin/0.log" Oct 30 19:49:10.403178 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:49:10.403137 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:49:11.143681 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:11.143652 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6944dd5f6b-8kpxt_7b4193ef-6fc1-497a-8706-ad093750a47b/kube-auth-proxy/0.log" Oct 30 19:49:12.060312 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:12.060275 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-dlrbh_177c6297-5b36-4a86-9baf-21793a0ec95d/istio-proxy/0.log" Oct 30 19:49:12.238391 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:12.238362 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-n5rx6_78d8bc24-bd69-4497-b7f8-cc98fcf3c379/istio-proxy/0.log" Oct 30 19:49:12.977098 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:12.977067 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-v7rds_e4d6a7ba-f41e-47b5-9e16-993a271bee50/istio-proxy/0.log" Oct 30 19:49:15.235351 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:15.235311 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-cn4sb_65b0be16-0564-48ea-85a8-48c7faa67535/main/0.log" Oct 30 19:49:15.414188 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:15.414159 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-bj75q_2d4749bf-c669-4371-af0f-e675731c3fc2/main/0.log" Oct 30 19:49:15.601300 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:15.601271 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-4qmpw_21a18edf-99e1-4344-b1e8-0e361f5d2c24/main/0.log" Oct 30 19:49:15.772323 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:15.772295 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp_93c33d89-f6fc-406f-928d-4819b28bbf73/main/0.log" Oct 30 19:49:15.951973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:15.951886 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-lvm6p_5a8648c2-b866-44c0-8371-e9223a8741eb/main/0.log" Oct 30 19:49:16.137905 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:16.137876 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj_8282cab2-7b52-4403-ac9e-a9897438dfba/main/0.log" Oct 30 19:49:16.326740 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:16.326713 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm_7eb64bc3-1b22-4944-b693-be0833371aaf/main/0.log" Oct 30 19:49:17.247469 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:17.247433 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c87c8b988-knzjq_408b1443-a9e2-493d-9597-1e5512dd88ff/authorino/0.log" Oct 30 19:49:27.114226 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:27.114194 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-zl5gt_88809f9c-f262-4f69-9a56-4876ed7404da/manager/0.log" Oct 30 19:49:28.171863 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:28.171831 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4cvck_72b21b6b-0d07-45e9-8899-b625ff8d5c5e/proxy/0.log" Oct 30 19:49:28.704903 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:28.704869 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5df7c6c68c-5zkbt_96e8227c-b7ae-45e7-b5bc-01562a8af474/maas-api/0.log" Oct 30 19:49:31.381829 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:31.381798 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c87c8b988-knzjq_408b1443-a9e2-493d-9597-1e5512dd88ff/authorino/0.log" Oct 30 19:49:31.971186 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:31.971157 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-vs77r_d631061b-03d9-4e53-a42d-f1ce92fc2af4/kuadrant-console-plugin/0.log" Oct 30 19:49:33.488316 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:33.488288 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6944dd5f6b-8kpxt_7b4193ef-6fc1-497a-8706-ad093750a47b/kube-auth-proxy/0.log" Oct 30 19:49:34.393237 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:34.393208 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-dlrbh_177c6297-5b36-4a86-9baf-21793a0ec95d/istio-proxy/0.log" Oct 30 19:49:34.588241 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:34.588213 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-n5rx6_78d8bc24-bd69-4497-b7f8-cc98fcf3c379/istio-proxy/0.log" Oct 30 19:49:35.307378 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:35.307347 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-v7rds_e4d6a7ba-f41e-47b5-9e16-993a271bee50/istio-proxy/0.log" Oct 30 19:49:36.934156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:36.934129 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-cn4sb_65b0be16-0564-48ea-85a8-48c7faa67535/main/0.log" Oct 30 19:49:37.115897 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:37.115871 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-bj75q_2d4749bf-c669-4371-af0f-e675731c3fc2/main/0.log" Oct 30 19:49:37.313288 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:37.313266 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-4qmpw_21a18edf-99e1-4344-b1e8-0e361f5d2c24/main/0.log" Oct 30 19:49:37.515571 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:37.515543 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8bbrxp_93c33d89-f6fc-406f-928d-4819b28bbf73/main/0.log" Oct 30 19:49:37.690666 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:37.690583 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-lvm6p_5a8648c2-b866-44c0-8371-e9223a8741eb/main/0.log" Oct 30 19:49:37.885856 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:37.885824 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-m2sfj_8282cab2-7b52-4403-ac9e-a9897438dfba/main/0.log" Oct 30 19:49:38.073447 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:38.073420 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-4njtm_7eb64bc3-1b22-4944-b693-be0833371aaf/main/0.log" Oct 30 19:49:38.519978 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:38.519944 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:39.579937 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:39.579912 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:49:39.582597 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:39.582568 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dvltm_4a3d3354-c4de-4bf4-8d99-80808fe2e6f9/ovn-acl-logging/0.log" Oct 30 19:49:40.404278 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:49:40.404242 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:49:40.432553 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:40.432524 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:46.675653 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:46.675615 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:49:50.560872 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:49:50.560793 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:50:10.403781 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:50:10.403740 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:50:40.062997 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:50:40.062960 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:50:40.402950 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:50:40.402859 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:50:46.721327 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:50:46.721273 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:50:59.236791 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:50:59.236758 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:06.351357 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.351323 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d8mgz/must-gather-8b8s5"] Oct 30 19:51:06.351855 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.351836 5627 cpu_manager.go:414] "containerMap: removing container" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" Oct 30 19:51:06.351910 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.351857 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" Oct 30 19:51:06.351951 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.351937 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" Oct 30 19:51:06.351951 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.351948 5627 memory_manager.go:365] "RemoveStaleState removing state" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" Oct 30 19:51:06.352018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.352007 5627 cpu_manager.go:414] "containerMap: removing container" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" Oct 30 19:51:06.352018 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.352014 5627 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42186c40-5e91-4467-8f25-6d5ad0fabdb8" containerName="istio-proxy" Oct 30 19:51:06.355073 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.355057 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8mgz/must-gather-8b8s5" Oct 30 19:51:06.360550 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.360524 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d8mgz\"/\"kube-root-ca.crt\"" Oct 30 19:51:06.360716 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.360557 5627 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d8mgz\"/\"default-dockercfg-8p9jx\"" Oct 30 19:51:06.360716 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.360567 5627 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d8mgz\"/\"openshift-service-ca.crt\"" Oct 30 19:51:06.363846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.363820 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:06.369744 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.369720 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8mgz/must-gather-8b8s5"] Oct 30 19:51:06.455940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.455914 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c98b8d7b-3f37-47c9-a1dc-554198876449-must-gather-output\") pod \"must-gather-8b8s5\" (UID: \"c98b8d7b-3f37-47c9-a1dc-554198876449\") " pod="openshift-must-gather-d8mgz/must-gather-8b8s5" Oct 30 19:51:06.455940 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.455941 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clnbz\" (UniqueName: \"kubernetes.io/projected/c98b8d7b-3f37-47c9-a1dc-554198876449-kube-api-access-clnbz\") pod \"must-gather-8b8s5\" (UID: \"c98b8d7b-3f37-47c9-a1dc-554198876449\") " pod="openshift-must-gather-d8mgz/must-gather-8b8s5" Oct 30 19:51:06.497392 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.497366 5627 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" limit=1024 Oct 30 19:51:06.556871 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.556852 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c98b8d7b-3f37-47c9-a1dc-554198876449-must-gather-output\") pod \"must-gather-8b8s5\" (UID: \"c98b8d7b-3f37-47c9-a1dc-554198876449\") " pod="openshift-must-gather-d8mgz/must-gather-8b8s5" Oct 30 19:51:06.556973 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.556876 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clnbz\" (UniqueName: \"kubernetes.io/projected/c98b8d7b-3f37-47c9-a1dc-554198876449-kube-api-access-clnbz\") pod \"must-gather-8b8s5\" (UID: \"c98b8d7b-3f37-47c9-a1dc-554198876449\") " pod="openshift-must-gather-d8mgz/must-gather-8b8s5" Oct 30 19:51:06.557208 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.557191 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c98b8d7b-3f37-47c9-a1dc-554198876449-must-gather-output\") pod \"must-gather-8b8s5\" (UID: \"c98b8d7b-3f37-47c9-a1dc-554198876449\") " pod="openshift-must-gather-d8mgz/must-gather-8b8s5" Oct 30 19:51:06.566427 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.566387 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clnbz\" (UniqueName: \"kubernetes.io/projected/c98b8d7b-3f37-47c9-a1dc-554198876449-kube-api-access-clnbz\") pod \"must-gather-8b8s5\" (UID: \"c98b8d7b-3f37-47c9-a1dc-554198876449\") " pod="openshift-must-gather-d8mgz/must-gather-8b8s5" Oct 30 19:51:06.669082 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.669023 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8mgz/must-gather-8b8s5" Oct 30 19:51:06.942226 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.942190 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8mgz/must-gather-8b8s5"] Oct 30 19:51:06.944359 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:51:06.944327 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc98b8d7b_3f37_47c9_a1dc_554198876449.slice/crio-d6f38473309fa383fad1c301775ab120f5b15b4f54f89be54145c8daf8c20513 WatchSource:0}: Error finding container d6f38473309fa383fad1c301775ab120f5b15b4f54f89be54145c8daf8c20513: Status 404 returned error can't find the container with id d6f38473309fa383fad1c301775ab120f5b15b4f54f89be54145c8daf8c20513 Oct 30 19:51:06.946397 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:06.946377 5627 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:51:07.803170 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:07.803083 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8mgz/must-gather-8b8s5" event={"ID":"c98b8d7b-3f37-47c9-a1dc-554198876449","Type":"ContainerStarted","Data":"6d1ba7aaffce6e03f161450deb5304d838acd6a5a66945802ece760a41c60133"} Oct 30 19:51:07.803170 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:07.803138 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8mgz/must-gather-8b8s5" event={"ID":"c98b8d7b-3f37-47c9-a1dc-554198876449","Type":"ContainerStarted","Data":"d6f38473309fa383fad1c301775ab120f5b15b4f54f89be54145c8daf8c20513"} Oct 30 19:51:08.814252 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:08.814214 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8mgz/must-gather-8b8s5" event={"ID":"c98b8d7b-3f37-47c9-a1dc-554198876449","Type":"ContainerStarted","Data":"5a35a231856f7b891326da7361a19486af2f4018fe3e95d545e744b615e8a501"} Oct 30 19:51:08.833209 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:08.833147 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-d8mgz/must-gather-8b8s5" podStartSLOduration=2.163169533 podStartE2EDuration="2.83313273s" totalImagesPullingTime="669.963197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:08.832915692 +0000 UTC m=+1889.559733002" watchObservedRunningTime="2026-10-30 19:51:08.83313273 +0000 UTC m=+1889.559950025" Oct 30 19:51:09.566088 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:09.566029 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f5lqj_52f645a5-26c7-4cad-9834-71eacd7c9a88/global-pull-secret-syncer/0.log" Oct 30 19:51:09.753918 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:09.753890 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6n7ll_756b72aa-da1a-4e51-8adf-df95022277d8/konnectivity-agent/0.log" Oct 30 19:51:09.973840 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:09.973758 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-211.us-east-2.compute.internal_1ce0953a30f3bb9b41a7629efbf90c90/haproxy/0.log" Oct 30 19:51:10.416029 ip-10-0-139-211 kubenswrapper[5627]: E1030 19:51:10.415989 5627 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9nxj" containerName="extractor" Oct 30 19:51:17.119726 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:17.113576 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c87c8b988-knzjq_408b1443-a9e2-493d-9597-1e5512dd88ff/authorino/0.log" Oct 30 19:51:17.217971 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:17.217938 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-vs77r_d631061b-03d9-4e53-a42d-f1ce92fc2af4/kuadrant-console-plugin/0.log" Oct 30 19:51:19.125243 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.125201 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjwlv_3f35fd1e-89aa-430a-8198-703bc598c411/node-exporter/0.log" Oct 30 19:51:19.157187 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.157153 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjwlv_3f35fd1e-89aa-430a-8198-703bc598c411/kube-rbac-proxy/0.log" Oct 30 19:51:19.194738 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.191764 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjwlv_3f35fd1e-89aa-430a-8198-703bc598c411/init-textfile/0.log" Oct 30 19:51:19.365345 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.365308 5627 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk"] Oct 30 19:51:19.370767 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.370737 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.386173 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.384625 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk"] Oct 30 19:51:19.540833 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.540799 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g49dk\" (UniqueName: \"kubernetes.io/projected/33e71cb6-0eb9-4395-a49b-6e2c900bb520-kube-api-access-g49dk\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.540999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.540861 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-podres\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.540999 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.540950 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-proc\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.541082 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.541003 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-lib-modules\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.541269 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.541246 5627 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-sys\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642203 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642109 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-sys\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642203 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642169 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g49dk\" (UniqueName: \"kubernetes.io/projected/33e71cb6-0eb9-4395-a49b-6e2c900bb520-kube-api-access-g49dk\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642436 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642205 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-sys\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642436 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642228 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-podres\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642436 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642280 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-proc\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642436 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642300 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-podres\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642436 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642356 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-proc\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642436 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642385 5627 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-lib-modules\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.642716 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.642446 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33e71cb6-0eb9-4395-a49b-6e2c900bb520-lib-modules\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.652495 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.652466 5627 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g49dk\" (UniqueName: \"kubernetes.io/projected/33e71cb6-0eb9-4395-a49b-6e2c900bb520-kube-api-access-g49dk\") pod \"perf-node-gather-daemonset-tdkhk\" (UID: \"33e71cb6-0eb9-4395-a49b-6e2c900bb520\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.670025 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.670001 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-ndcj8_14ef0160-1e62-4a92-8158-bdd45eebaeab/prometheus-operator/0.log" Oct 30 19:51:19.690474 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.690446 5627 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:19.696130 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.696105 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-ndcj8_14ef0160-1e62-4a92-8158-bdd45eebaeab/kube-rbac-proxy/0.log" Oct 30 19:51:19.765156 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.765133 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dc5d5fc9c-htqfc_df041cfc-c4b4-4ee1-bc4e-c79605faaa97/telemeter-client/0.log" Oct 30 19:51:19.818630 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.818597 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dc5d5fc9c-htqfc_df041cfc-c4b4-4ee1-bc4e-c79605faaa97/reload/0.log" Oct 30 19:51:19.863456 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.863429 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dc5d5fc9c-htqfc_df041cfc-c4b4-4ee1-bc4e-c79605faaa97/kube-rbac-proxy/0.log" Oct 30 19:51:19.991846 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:19.991557 5627 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk"] Oct 30 19:51:19.994530 ip-10-0-139-211 kubenswrapper[5627]: W1030 19:51:19.994504 5627 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod33e71cb6_0eb9_4395_a49b_6e2c900bb520.slice/crio-36d6b0208ab5e4bb21794e01e43b0b0b1e99435fa3c4bdd62e5277ba4994dd1b WatchSource:0}: Error finding container 36d6b0208ab5e4bb21794e01e43b0b0b1e99435fa3c4bdd62e5277ba4994dd1b: Status 404 returned error can't find the container with id 36d6b0208ab5e4bb21794e01e43b0b0b1e99435fa3c4bdd62e5277ba4994dd1b Oct 30 19:51:20.921728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:20.919448 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" event={"ID":"33e71cb6-0eb9-4395-a49b-6e2c900bb520","Type":"ContainerStarted","Data":"54baa67b8a3d9246c3fba6dc839a4bd9952b6c735a940940f4857f26341e4a1d"} Oct 30 19:51:20.921728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:20.919525 5627 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" event={"ID":"33e71cb6-0eb9-4395-a49b-6e2c900bb520","Type":"ContainerStarted","Data":"36d6b0208ab5e4bb21794e01e43b0b0b1e99435fa3c4bdd62e5277ba4994dd1b"} Oct 30 19:51:20.921728 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:20.919686 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:20.940493 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:20.940447 5627 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" podStartSLOduration=1.940433804 podStartE2EDuration="1.940433804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:51:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:20.939043076 +0000 UTC m=+1901.665860376" watchObservedRunningTime="2026-10-30 19:51:20.940433804 +0000 UTC m=+1901.667251113" Oct 30 19:51:22.564908 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:22.564879 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5ff56bc7d5-mq2fx_6660566b-b2ba-4443-b892-f58b3384db5d/console/0.log" Oct 30 19:51:24.512380 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:24.512355 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-79mtg_acf1ec3f-be10-4166-ba73-8994e5d8d7a1/dns/0.log" Oct 30 19:51:24.540052 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:24.540015 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-79mtg_acf1ec3f-be10-4166-ba73-8994e5d8d7a1/kube-rbac-proxy/0.log" Oct 30 19:51:24.716504 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:24.716469 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cgl7t_eb0fad83-891c-4219-a590-846d4ca0089e/dns-node-resolver/0.log" Oct 30 19:51:25.470993 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:25.470961 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vvzd7_a008f757-b4b8-48a7-bb30-ffa1c1d39b75/node-ca/0.log" Oct 30 19:51:26.937925 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:26.937898 5627 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-tdkhk" Oct 30 19:51:28.092484 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:28.092450 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6944dd5f6b-8kpxt_7b4193ef-6fc1-497a-8706-ad093750a47b/kube-auth-proxy/0.log" Oct 30 19:51:28.301488 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:28.301452 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-dlrbh_177c6297-5b36-4a86-9baf-21793a0ec95d/istio-proxy/0.log" Oct 30 19:51:28.337375 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:28.337348 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-n5rx6_78d8bc24-bd69-4497-b7f8-cc98fcf3c379/istio-proxy/0.log" Oct 30 19:51:28.490801 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:28.490776 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-v7rds_e4d6a7ba-f41e-47b5-9e16-993a271bee50/istio-proxy/0.log" Oct 30 19:51:30.301042 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:30.301008 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lndpb_a5fac3e1-506c-4ce0-b992-9108addc1e81/serve-healthcheck-canary/0.log" Oct 30 19:51:31.291429 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:31.291388 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6944dd5f6b-8kpxt_7b4193ef-6fc1-497a-8706-ad093750a47b/kube-auth-proxy/0.log" Oct 30 19:51:31.422606 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:31.422576 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-dlrbh_177c6297-5b36-4a86-9baf-21793a0ec95d/istio-proxy/0.log" Oct 30 19:51:31.451326 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:31.451299 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-n5rx6_78d8bc24-bd69-4497-b7f8-cc98fcf3c379/istio-proxy/0.log" Oct 30 19:51:31.573394 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:31.573302 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-v7rds_e4d6a7ba-f41e-47b5-9e16-993a271bee50/istio-proxy/0.log" Oct 30 19:51:31.640141 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:31.640111 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w9nxj_6afdc8a2-0be6-424f-ba13-99a3fcf436dc/kube-rbac-proxy/0.log" Oct 30 19:51:31.704968 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:31.704942 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w9nxj_6afdc8a2-0be6-424f-ba13-99a3fcf436dc/exporter/0.log" Oct 30 19:51:31.753301 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:31.753242 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w9nxj_6afdc8a2-0be6-424f-ba13-99a3fcf436dc/extractor/0.log" Oct 30 19:51:34.799144 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:34.799116 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-668f68f55d-vpdg7_7bb0eda9-94df-4613-acba-a0ab4f0bb9a2/manager/0.log" Oct 30 19:51:34.828409 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:34.828377 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-668f68f55d-vpdg7_7bb0eda9-94df-4613-acba-a0ab4f0bb9a2/copy-manifests/0.log" Oct 30 19:51:34.982828 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:34.982782 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6944dd5f6b-8kpxt_7b4193ef-6fc1-497a-8706-ad093750a47b/kube-auth-proxy/0.log" Oct 30 19:51:35.091453 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:35.091381 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-dlrbh_177c6297-5b36-4a86-9baf-21793a0ec95d/istio-proxy/0.log" Oct 30 19:51:35.112237 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:35.112210 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-n5rx6_78d8bc24-bd69-4497-b7f8-cc98fcf3c379/istio-proxy/0.log" Oct 30 19:51:35.192869 ip-10-0-139-211 kubenswrapper[5627]: I1030 19:51:35.192840 5627 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-v7rds_e4d6a7ba-f41e-47b5-9e16-993a271bee50/istio-proxy/0.log"